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Vocational Assessment and Evaluation Systems 

A Comparison 



Karl F. Botterbusch, Ph.D., CVE 
Materials Development Center 
^^ocational Rehabilitation Institute 

diversity of Wisconsin-Stout 
Menomonie, Wisconsin 54751 



Preface 



This publication will be the fifth MDC publication since 1976 offering an objec- 
tive comparison of commercial vocational and assessment systems. Beginning in 1976 
with A Comparison of Seven Vocational Evaluation Systems , continuing with A Com- 
parison of Four Vocational Evaluation Systems in 1977, A Comparison of Commercial 
Vocational Evaluation Systems in 1980, and most recently the 1982 publication of A 
Comparison of Commercial Vocational Evaluation Systems (second edition), this series 
of publications has sought to provide vocational evaluators, special educators, voca- 
tional educators, manpower specialists, corrections personnel, and private rehabilitation 
practitioners with accurate and detailed information on widely availalle vocational 
evaluation and assessment systems. 

Although it has only been four years since the 1982 Comparisons, the present 
publication reflects the significant changes that occurred in evaluation and assessment 
technology. Within the assessment industry there have been two major changes. 
First, the last four years have seen the development of six new systems (i.e., Ap- 
ticom, KEVAS, MESA, MECA, SAM and WOWI), all of which are partially or entirely 
dependent on computers for either administration or scoring. Second, with the excep- 
tion of the MECA, all of these systems are assessment systems, not evaluation sys- 
tems. 

The development of this new generation of instruments is the result of two 
forces. First, due mostly to reductions in rehabilitation funding, many state vocation- 
al rehabilitation agencies have reduced the amount of time allowed for vocational 
evaluation from two or three weeks to one week or less. In the past several years, 
evaluators have had to cope with the problem of obtaining accurate information about 
clients in ever shorter time periods. The new assessment systems relying on psycho- 
logical testing and isolated trait work samples have offered one viable option. In 
addition, the strategy of many rehabilitation professionals, especially the ever increas- 
ing number in private practice, has changed from one of a lengthy vocational evalua- 
tion, usually followed by work adjustment and skill training prior to placement to one 
of direct placement after assessment or evaluation. This forces the evaluator to deal 
less with the client's ultimate potential and more with the direct transfer of skills 
and aptitude to currently available jobs. 

The second force is the passing of the Carl Perkins Act (PL 94-142). This law 
mandates the assessment of all handicapped students in public schools. This single act 
has done more to expand the market for evaluation and assessment products than all 
other factors combined. It has also shifted the major market for evaluation products 
from vocational rehabilitation to education. All of the new work sample systems 
mentioned above were developed mainly for an education market, not vocational 
rehabilitation. 

Based on changes within the evaluation industry and changes outside that in- 
dustry, there are several recent trends that show no sign of weakening in the future. 
For both profit and non-profit rehabilitation programs, there will be the continued 
emphasis on assessment at the expense of vocational evaluation. If this continues, it 
is likely that the job sample and simulated work samples that have been the standard 
for over 15 years will be largely replaced by isolated trait work samples and psycho- 
metric instruments. This will continue until it is realized that many severely disabled 
persons cannot be evaluated in a few hours, resulting in a movement for additional 
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time, and more client or student involvement in the evaluation process. Then the 
cycle from long to short time periods will begin anew. 

Another related trend has been the development and successful marketing of 
computerized job matching systems. Most of these systems require the entry of a 
client Worker Trait Profile, based on U.S. Department of Labor job analysis concepts, 
such as found in the Dictionary of Occupational Titles : this need for standardized 
traits may restrict or force many evaluation and assessment systems to initially plan 
or redesign the systems around the evaluation or assessment of DOL traits. 

I would like to thank the commercial developers who have willing provided 
manuals, technical reports, forms, and hardware for their respective systems. Finally, 
thanks to Darlcne Botterbusch who designed the format, entered the text and edited 
this monograph. 



Karl F. Botterbusch, Ph.D., CVE 
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Some Considerations for Selecting Commerciai 
Vocational Evaluation Systems 



This short section considers two separate models for the sclectioa of vocational 
evaluation or assessment systems. This first is the initial development of an evalua- 
tion unit; the second deals v/ith the expansion of an existing unit. 

Starting a New Evaluation Unit 

Faced with the need to equip and administer a new vocational evaluation or 
assessment unit, many educators and evaluators believe that the purchase of a single 
commercial evaluation system will solve their problems. The evaluator should first 
analyze a number of factors before deciding how to equip the unit and then carefully 
investigate all the techniques (i.e., work samples, psychological testing, situational 
assessment, and job site evaluation) to determine the ones that provide him/her with 
the best methods of adequately assessing his/her clients. It is assumed that the main 
goal of most evaluations is to obtain accurate and useful data about a student or 
client in the shortest time period. The emphasis is on the word "accurate." Persons 
with no or slight disabilities may be accurately as-essed using standardized psycho- 
logical testing or one of the new assessment batteries (e.g., SAGE, CES, Apticom or 
MESA) in a few hours. Other more severely disabled persons may take several days 
or even weeks (e.g., JEVS, McCarron-Dial and VIEWS). Thus, while accuracy must be 
the main goal, it will take longer and will be more difficult to evaluate some students 
or clients than it will others. 

A secondary goal is often to provide the client or student with specific voca- 
tional information, such as his/her interest in specific occupational areas, a realistic 
knowledge of his/her vocational strengths and weaknesses, specific vocational goals, 
and a plan for achieving these goals. Much of this information is provided through 
the honest and realistic sharing of results with the client, regardless of the evaluation 
tools used. It does imply, however, that the client has the opportunity to explore 
occupations. This can be provided through print and media programs, visits to train- 
ing programs or industry, job site evaluation and work samples. The VES, Prep Work 
Samples and the new MECA are three work sample systems offering a wide variety of 
simulated work experiences. In summary, the two most common goals of any voca- 
tional evaluation unit are: (1) to accurately and quickly determine the client's abili- 
ties, capacities, etc. and (2) to provide the client with realistic information about 
him/herself. 

These goals must then be related to the realistic work and/or training oppor- 
tunities that exist beyond the evaluation unit. There are three considerations. The 
first consideration is the client population. Some evaluation nnits must be capable of 
serving clients with all types of mental, physical, psychological, and cultural dis- 
abilities. Other facilities and/or schools restrict themselves to serving either a single 
disability or a small number of disabilities. An evaluation unit dealing with many 
types of handicaps would generally need to have techniques covering the entire range 
of occupational areas and skill levels within these areas. A unit providing services to 
a single disability group can safely limit its evaluation areas. For example, a facility 
or school serving only mentally retarded clients could realistically avoid evaluation for 
occupations requiring large amounts of formalized training or higher education. Some 
work sample systems claim to have been designed specifically for a particular level of 
student or client functioning. When selecting evaluation tools, keep in mind the type 
of client(s) served; it would be a waste of time to assess a client for a job which 
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he/she could not fill because of his/her handicap. At present, most commercial 
vocational evaluation systems are designed for persons who can see and hear and who 
can understand written and/or spoken English; most contain no specific instructions or 
modifications for persons with significant hearing or sight loses. The evaluator should 
be aware that he/she will frequently have to modify commercial work samples so they 
meet the special needs of his/her clients (Botterbusch & Menz, 1986). In summary, if 
an evaluator is considering a commercial evaluation or assessment system, he/she 
should check the battery against the needs of the client population served and then 
decide: (1) whether the system is designed for the target disability group(s), or (2) 
whether other evaluation techniques would be more appropriate. 

The second is the relationship between the community and the vocational evalua- 
tion unit. The teacher or evaluator must carefully investigate the range and type of 
jobs available in the local labor market. Thus, a small rural school or facility or a 
unit in a one industry community will have a more narrow range of job evaluation 
stations than either a urbau facility or a secondary school with a large number of 
vocational training programs. Labor market information can be obtained through job 
surveys, local employment offices and agencies, and student placement records. Once 
potential employment opportunities have been determined, intelligent decisions can be 
made on what type of evaluation tools can best assess these demands. 

The third consideration is training opportunities. Because the evaluation out- 
comes may not result in immediate placement, it is also necessary to investigate 
training opportunities. Even if the evaluation unit is located in a school, the evalu- 
ator needs to also consider training programs beyond a particular school or school 
system. The training options should be reflected in the selection of evaluation tools. 
A client's or student's range of occupations widens and his/her chances for upward 
mobility are frequently increased as a result of training. The presence of an area 
vocational-technical school, private trade and business schools, on-the-job training 
programs, apprenticeship programs, and higher education should be reflected in the 
evaluation unit. Vocational evaluation techniques covering a wide variety of occupa- 
tional areas and assessing the full range of client/student aptitudes and interests are 
needed if the facility is in an area where many employment and training opportunities 
exist. 

Based on the discussion above, the evaluator should take the following specific 

steps: 

1. Decide on the goal(s) of the evaluation unit. 

Example: Colfax High School has a large number of students that must be 
evaluated under the Carl Perkins Act. In addition to this special population, 
the school guidance counselor would like to use the evaluation unit for all 
students who need to set some specific vocational goals. From this descrip- 
tion, an evaluation unit will have several goals: (1) to provide accurate 
assessment of aptitudes, abilities, and literacy for a special population; (2) to 
provide accurate assessment of aptitudes, abilities and literacy for a general 
student population; (3) to provide occupational information for all students 
commensurate with their abilities; and (4) to offer occupational experiences 
for all students to provide a basis for realistic self-knowledge. 



It must be noted that if a school or other institution plans to establish a com- 
plete evaluation unit, they will most likely Nave to consider purchasing more than one 
commercial system. Because different systems are designed to meet different needs, it 
is unlikely that one system will solve all evaluation problems. 

Changing an Existing Vocational Evaluation Unit 

Like everything else in human services and education, vocational evaluation and 
assessment can change, often suddenly. With regard to vocational evaluation and 
assessment there are two major changes: (1) changes in type of client served and (2) 
changes in the community labor market. Evaluation unit equipment and procedures 
must be updated to reflect *hese changes. Often the unit finds that the composition 
of the clients served has changed. Some of these changes are anticipated, others are 
planned for, and yet others occur because of shifts in funding or program decisions 
beyond the control of the evaluation unit. When significant changes occur in the 
type of clients served, the evaluator must carefully review the work samples and 
other assessment techniques to determine the relevance of present evaluation proce- 
dures. For example, the decision to provide evaluation services to high school age 
students could result in an increase in the number and type of vocational exploration 
work samples used. A decision to serve industrially injured workers would place more 
emphasis on physical capacity assessment. 

The second consideration is changes in the local job market and local training 
opportunities available to the clients. The evaluation unit is seen as part of the 
community and changing to accommodate the changing needs of that community. The 
following process is suggested as one means of keeping up with change: 

First, carefully determine exactly what the changes are and the degree of their 
impact on the evaluation unit. For example, if the unit receives refe-rals from a new 
source or a sudden increase in numbers of a particular disability group, then the 
center needs to find out the reasons and adjust accordingly. 

Second, the community should be carefully monitored for major shifts in the type 
of jobs available. Employment Service records, articles in the business section of 
local papers, personal contacts, and other sources should be used to determine chan- 
ges. Some of these changes could be the opening of a nev/ plant, the closing of a 
heavy manufacturing industry, an increase in new housing and other construction, the 
development of a new shopping mall, and expansion of an existing industry. These 
shifts should be carefully noted. 

Third, the evaluation unit is changed to reflect current employment needs. The 
specific assessment tools used are changed. Thus, if the employment base of the 
community has changed from heavy manufacturing to light manufacturing and service, 
the evaluation unit should add more work samples in the business service areas. In 
short, the evaluation un:t must reflect the labor and educational markets of the 
community. 

Once the current labor and educational market are known, the evaluator needs to 
select work samples and other assessment devices that assess for and provide occupa- 
tional information about these jobs. The traits measured by each individual work 
sample or system should be compared to the information gained about the community. 



Vocational Evaluation System Outline 



1. Development 

a. Sponsor - The organization that originally developed the vocational evaluation 
or assessment system. Public funding sources, when applicable, are noted. 

b. Target Group - What specific populations, such as disadvantaged, mentally 
retarded or physically disabled, was the systenr. designed to serve? The 
purpose or purposes of the vocational evaluation system (e.g., vocational 
assessment, occupation exploration) as seen by the developer and reviewer are 
included. 

c. Basis of the System - What theoretical or organizational principle, such as 
the Dictionary of Occupational Titles, was used as a basis for development. 

2. Organization 

a. Name and Number of Work Samples - How many work samples does the 
system contain, and what are their names? 

b. Grouping of Work Sample - What is the arrangement of the individual work 
samples within the system? Are several work samples grouped Jn a hierarchy, 
classified according by DOT code, or is each work sample independent? 

c. Manual - What is the organization and contents of the manual(s)? Does it 
provide all the details that the evaluator needs to know in order to set-up, 
administer, score, and interpret the worJ' samples? 

3. Physical Aspects 

a. Packaging of the Work Samples - How are work samples packaged for sale 
and use? Is each work sample self-contained or must tools and equipment be 
shared with other work samples? 

b. Durability - How durable are the tools and equipment in the system? If the 
system uses audiovisual components, how prone to breakdown are they? 

c. Expendable Supplies - How much and what type of expendable supplies (e.g., 
wood, paper, cloth) are need.'Jd per client? 

d. Replacement - To what degree can supplies and materials (e.g., tools, nuts 
and bolts, colored chips) be obtained locally or must they be ordered from 
the developer? How does such replacement reHect on the standardization of 
the evaluation system? 

c. Computer Requirements - If the system requires the use of a computer for 
work sample administration and/or scoring, what are the computer system's 
requirements? What type of software is used? 
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Vocational Evaluation Process 



a. Preliminary Screening - What information is needed or what decisions must be 
made before a client can be administered the system? For example, does the 
students's reading level first need to be determined, is the system only in- 
tended for certain types of disability groups and what skills are needed prior 
to beginning testing? 

b. Sequence of Work Sample Administration - In what order, if any, are the 
work samples administered? Are they administered by increasing skill le/el? 
Are they administered by interest area? What is the usual order of admin- 
istration? What variations are permitted? Do all work samples have to be 
administered? 

c. Client Involvement - To what extent is the client informed of his/her prog- 
ress during the course of work sample administration? What type, if any, 
formal feedback is given to the student or client after the entire battery has 
been administered? What type of contact does the client have with the 
evaluator? Does the student or client play an active part in the evaluation 
process? 

d. Evaluation Setting - Does the general environment attempt to simulate com- 
petitive employment conditions, produce a classroom atmosphere, or resemble 
a formal testing situation? 

e. Time to Complete the Entire System - How many hours or days does it take 
the average person to complete all the work samples in the system? 

Administration 

a. Procedures - Are the purposes of each work sample, materials needed, layout, 
timing, instructions to the client, etc. clearly given so that there is little 
chance of incorrect administration? 

b. Method of Instruction Giving - How does the student or client receive his/her 
instructions for the work samples in the system? For example: oral, demon- 
stration, written instructions or computer? Does the work sample manual 
permit variations in administration procedures? 

c. Separation of Learning/Performance - Does the work sample have separate 
practice (i.e., learning) and performance (i.e., production) periods? Are there 
definite criteria (e.g., three consecutive correct assemblies; lines drawn within 
+ or - 1/16 inch) that the student or client must reach before he/she can 
progress from a practice period to a performance phase? 

d. Providing Assistance to the Client - What procedures are there for giving 
extra or additional instructions, demonstrations or feedback after initial 
instructions? Can the evaluator or teacher offer advice or help during the 
performance phase of the work sample? 

e. Repeating Work Samples - What provisions are made for readministration of 
some work samples and what is the purpose of readminstration? Are work 
samples repeated until the student or client reaches a predetermined level? 
Are readministrations plotted on a chart to produce a performance curve? 

7 
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6. Scoring and Norms 

a. Timing - What are the procedures for timing the client? 

b. Timing Interval - At exactly what point does the evaluator start timing the 
student and when does he/she stop? Are there specific cut-offs or does the 
student or client continue until the work sample is completed? 

c. Time Norms - What is the procedure for reporting the time score for each 
work sample? Are norms given in percentiles, a rating, percent of industrial 
normal, etc.? 

d. Error Scoring - What procedures, such as a random check of some parts, 
general rating of overall quality or a comparison to standards, are used to 
determine errors? 

e. Scoring Aids - What use is made of overlays, templates, models, etc. to make 
error scoring more accurate and easier for the evaluator? 

f. Quality Norms - What procedures are used for reporting the number of errors, 
quality ratings, etc., for each work sample? 

g. Emphasis in Scoring - Does the system emphasize time or errors in the 
scoring process or are both given equal weight? 

7. Observation of Clients 

a. Work Performance - Are work performance factors (e.g., fine finger dexterity, 
color perception) listed for the system in general and are specific work 
performance factors given for each work sample? 

b. Work Behaviors - Are work behaviors (e.g., ability to follow instructions, 
communication with supervisors) defined for the system and are specific work 
behaviors to be observed for each work sample? 

c. Recording System - What procedures does the system have for the recording, 
describing and rating of observed work performance and work behaviors? 

d. Frequency of Observation - How often (e.g., every five minutes) and to what 
extent is the evaluator or teacher to observe and record client behavior? 
What type of sampling, if any, is used, such as time sampling or event sam- 
pling? 

8. Reporting 

a. Forms - What forms for recording time and quality, work performance, work 
behavior, etc. are used for each work sample in the system? What are the 
major content areas of these forms? 

b. Final Report Format - What information is included in the final report and 
what type of format (e.g., rating scales, free narration) is used to present 
this information? Is the final report format and/or an example report given 
in the work sample manual? 
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9. Utility 



a. 



Vocational Exploration - Does the sysrem provide experiences that the client 
or student can readily relate to competitive employment? 

b Vocational Recommendations - Are training and job recommendations specific 
or general? How are they related to the DOT or other job classification 
systems? Can additional evaluation, work adjustment, skill training, direct 
placement or other services be recommended as a result of using the system? 

c. Counselor Utilization - Can the system provide the counselor or referring 
agency with useful information either in the form of a final report and/or 
recommendations? 

10. Training ip the System 

a. Training Required - Is formal training required before the system is sold? 

b. Training Available - Is formal training available? Where is it available? 

c. Duration - How much time is required for training? 

d. FoIIow-up - Is technical assistance available after purchase and training? 

11. Technical Considerations ■ 

a. Norm Base - On what types of populations (e.g., client, employed, general 
working population) was the system normed, and are these norm groups 
clearly defined? Are norm groups of adequate size for practical use? Are 
predetermined time standards, such as Methods-Time-Measurement (MTM), 
used? 

b Reliability - What empirical evidence is there to demonstrate that the system 
and its component work samples give reliable or consistent results? Are the 
research methods, sample sizes, sample characteristics and statistical proce- 
dures described in enough detail for the user to judge the meaningfulness of 
the results? 

c Validity - What content, construct or empirical validity data are available to 
indicate that the system or individual work samples within the system really 
do°s what it claims, such as make more realistic vocational choices or predict 
job and/or training s.iccess? Are research methods, sample sizes, statistics 
etc. described in enough detail to permit the user to judge the usefulness of 
the system for his/her evaluation unit or school? 

12. Reviewer's Summary and Comments 

This contains what the reviewer considers to be the major advantages and disad- 
vantages of the system. Also included are any unique points about each system 
and some ideas for its use. 




1 In general, this section relates each system to the Standa rds for Educat-.on^l 
and PsycholQgicai Testing (American Educational Research Association, et al., 1985). 
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13. Address 

The national home address of the company or institution selling the system. 

14. Cost 

The cost of the system at the time of publication and what materials and services 
are included in the price. 

15. References 

All generally available, non-promotional, references are given. 
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OUTLINE 


APTICOM 


CES 


1. Development 

a. Sponsor 

b. Target Group 

c. Basis of System 


JEVS 

disadvantaged students, special educa- 
tion, rehabilitation clients 

U.S. DOL Aptitudes 


Rehabilitation Services Administration 
general population, disabled 

Data-People -Things 


2. Organisation 

a. Number of Work Samples 

b. Grouping of Work Samples 

c. Manual 


14 tests and work samples 
10 Aptitudes, GED, Interests 

r^ministration and technical manuals; 
very well- writ ten 


28 or less tests and apparatus tests 

3 series of tests for different popula- 
tions 

separate manual for each series; all 
system details 


3. Physical Aspects 

a. Packaging 

b. Durability 

c. Expendable Supplies 

d. Replacement 

e. Computer Requirements 


system self-contained 

ftdrly durable with careful use 

forms and orinter paper 
from developer 

dedicated computer included; can be 
downloaded to IBM-PC 


packaged separately for shipping 

fairly durable; needs recalibration 

forms and tests 
from developer 

optional IBM, Apple; modem; 228K; 2 
disk drives 


4. Vocational Evaluation Process 

a. Preliminary Sci*eening 

b. Sequence of Work Sample 
Administration 

c. Client Involvement 

d. Evaluation Setting 

e. Time to Complete Entire 
System 


not mentioned 

tests given in set sequence 

very little 

creates formal testing atmosphere 
1 1/2 to 2 hours 


not required 
any order 

very little 

creates formal testing atmosphere 
differs with series (3 1/2-4 hours) 


5. Administration 
a. Procedures 

b Method of Instruction Giving 

c. Separation of Learning/ 
Performance 

d. Providing Assistance to 
Client 

e. Repeating Work Samples 


specified in detail 

oral Sc demonstration, limited reading 
minimal 

not specified 

not repeated 


specified in detail 

oral Sc demonstration, limited hearing 
clear separation 

no assistance after timing begins 
if invalid results suspected 
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KEVAS 


McCarron-Dial 


MESA 


Key Education, Inc. 

himdicapped & "normal" populations, 
students 

basic elements of psychophysical 
functioning 


McCarron^Dial 

mentally retarded, mentally ill, learn- 
mg disabled 

5 neuro-psychological factors 


Valpar 

middle 80% of population 
DOT Worker Trait Group 


20 tests and apparatus tests 

3 groups: psychophysical, attitudes, 
and work related competencies 

general manuals, contains most system 
details 


6 basic mrasures; 5 optional 
5 factor 

several manuals, contains all system 
details 


84 separate pieces of information 
9 major test categories 

general manual, contain^, all system 
details 


system self-contained 

fairly durable 

forms and tests 
from developer 
no local computer needs 


packaged into three kits 

fairly durable 

forms and answet sheets 
from developer 

optional IBM-PC, Apple. TRS-80 


computer equipment used for several 
tests 

durable tests; computer equipment 
less durable 

forms and tests 

from developer 

Apple, IBM; 1 disk drive; printer; 
special equipment 


not mentioned 

suggested order given in manual 

very little 
laboratory 
2 1/2 to 3 hours 


client interviewer 
factors 1 through 5 

encouraged 

formal testing and workshop 
5 days with behavior observation 


not required 

suggested sequence given in manual 

very little 
informal testing 
A hours 


problems with format 

oral Sc demonstration, limited reading 

minimal 

careful monitoring on apparatus tests 
not speciHed 


specified in detail 
oral Sc demonstration 
not applicable 

little assistance provided 

if invalid results suspected 


specified in detail 

oral, demonstration, computer given 
minimal 

additional instructions provided 
permitted 
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OUTLINE 



APTICOM 



6, Scoring tmd Norms 

a. Timing 

b. Timing Interval 

c. Time Norms 

d. Error Scoring 

e. Scoring Aids 

f. Quality Norms 

g. Emphasis in Scoring 



7. Observation of Clients 

a. Work Performance 

b. Work Behaviors 

c. Recording System 

d. Frequency of Observation 



8. Reporting 

a. Forms 

b. Final Report Format 



9. Utility 

a. Vocational Exploration 

b. Vocational Recommendations 

c. Counsel Utilisation 



10. Training in the System 

a. Training Required 

b. Training Available 

c. Duration 

d. Tollow-up 



11. Technical Considerations 

a. Norm Base 

b. Reliability 

c. Validity 



preset times for each test 

after end of practice session 
no time norms used 
number of correct responses 
computer scored 
not relevant 

number of correct responses 



CES 



evaluator times client, electrical 
devices 

varies with type of tests 
no separate time norms 
number of correct responses 
some use 
not relevant 

number of correct responses 



no client observations made 

no client observations made 

no client observations made 
no client observations made 



3 profile sheets for hand scoring 
several options 



very limited 

gives work group and job matches 
specific information given 



no 

yes - video tape 
not mentioned 
not mentioned 



selected samples; adequate siee 
adequate data in manual most in .80's 
correlations with U.S. DOL tests 
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no factors observed 

limited information 

5 point rating scale; people functions 
not relevant 

standard data entry form 
single computer generated form 



very limited 

lists specific job titles 

dependent upon user 



yes 

yes audio cassette 
12 hours 
available 



under development 
test-*retest adequate 
very limited 



KEVAS 


McCarron-Dial 


MESA 


tests not timed 

not relevant 

no separate time norms 

number cf correct responses 

not used 

not relevant 

number of correct responses 


evaluator time; client 

speciHed time limits 
some separ&te time norms 
compared to standards 
not used 

combined with time norms to form 
single overall score 

quality and quantity 


computer timed 

several methods 
no time norms used 
not used 
not used 
not used 

number of correct responses 


no factors observed 

limited information 

2 point rating scale 
not specified 


specific factors defined 

specific factors defmed 

S and 5 point scale 
2 hours for 5 days 


specific work performance and work 
behaviors not used 

specific work performance and work 
behaviors not used 

some tests scored through observation 
constant 


standard forms for all phases 
several computer scored options 


standard forms for all phases 

profile of results and recommenda- 
tions 


standard forms for all phases 
several options, computer generated 


very limited 

gives specific jobs ic training pro- 
grams 

notes impairments 


very limited 

level of functioning 

oriented towards counselor 


very limited 
specific test results 

worker trait profile - compare to 
jobs 


yes 
yes 

2 days 

yearly recertiHcation required 


yes 
yes 

S days 

advanced training recommended 


no 
yes 

2 days 

advanced training available 


3 national normative studies 
several methods most in .80*8 
two studies available 


extensive 

mostly test-retest .80's-.90's 
extensive as diagnostic instrument 


MTM students Sc employed workers 
test-retest .79-.96 
mostly construct 
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M£<C A 


Micro- 1 UWLK 








1. Development 






ft. Sponsor 


Cor over Conspany 


ICD International Center f/t Disabled 


b. Target Group 


special needs students 


general rehabilitation population 


c. Basis of System 


work groups 


DOT and GATB 


2. Organisation 






t Number of Work Samples 


15 


13 


b. Grouping of Work Samples 


each is independent^ 


5 aptitude areas 


c. Manual 


separate manual for each work sample 
completed 


general manual, separate manual for 
each work sample 


3. Physical Aspects 






a. Packaging 


each packaged separately 


each packaged separately 


b. Durability 


durable 


durable 


c. Expendable Supplies 


electronics, paper, cleaning supplies, 
etc., forma 


wire only 


d. Replacement 


from developer 


forms locally duplicated 


e. Computer Requirements 


Apple; 1 disk drive 


not applicable 


4. Vocational Evaluation Process 






a. Preliminary Screening 


not specified 


not required 


b. Sequence of Work Sample 
Administration 


any order 


discretion of evaluator 


c. Client Involvement 


some 


extensive client involvement 


d. Evaluation Setting 


clarsroom 


formal testing ic counseling combina- 
tion 


e. Time to Complete Entire 
System 


average 2 hours per work sample 


15-20 hours 


5. Administration 






a. Procedures 


specified in detail 


specified in detail 


b. Method of Instruction Giving 


computer and audio casset«,e 


audio cassette, evaluation demonstra- 
tion 


c. Separation of Learning/ 
Performance 


none 


stressed, almost total 


d. Providing Assistance to 
Client 


^valuator check points 


no assistance after timing begins 


e. Repeating Work Samples 


not specified 


not speciHed 
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Vnlnflr 17 


U.S. Department of Labor 


PREP 


Valpar International 


disadvantaged, special needs 


special needs, students, manpower 


mentally retarded 


DOT & GOE 


Career Clusters & DOT 


not specified 


28 


27 


11 assessment techniques using dif- 
ferent formats 


12 work groups 


each is independent 


5 areas 


single manual contains all system 
details 


general and separate manuals contain 
all details 


general manual, separate manual for 
each, contains all details 


each packaged separately 


each packaged separately 


each of 5 areas packaged separately 


very durable 


durable 


very durable 


fabric, paper, metal, string 


wood, circuit boards, food items, etc., 
forms 


no consumable materials 


most purchased locally 


purchased locally, forms from devel- 
oper 


forms locally duplicated or order from 
developer 


not applicable 


not applicable 


not applicable 


not required 


not required 


one section given as screening 


progression from least to most com- 
plex 


any order 


any order 


some 


extensive client involvement 


considerable 


realistic work setting stressed 


cl&ssr«)om atmosphere 


creates formal testing situation 


6-7 days 


average - 2 hours per work sample 


5 1/2 hours 


specified in detail 


speciHed in detail 


speciHed in detail 


oral & demonstration 


audiovisual 


oral Sc demonstration, follow sample 


minimal 


minimal 


minimal 


assistance lowers scores 


little assistance after timing begins 


not specified 


not recommended 


usually not readministered 

1 


strongly recommended 
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OUTLINF 


MFC A 


Micro-TOWER 
















sepaziibO bimes lor eacii cosjc 




c. Timtt Norms 


rated on 5 point scale 


no time norms used 


d. Error Scoring 


major & minor errors defmed 


number completed, pieces correct 


e. Scoring Aids 


not used 


some use 


f. Quality Norms 


rated on 5 point scale 


MODAPTS, rated on 5 point scale 


g. Emphasis in Scoring 


time ic quality given equal weight 


quality 


7. Observation of Clients 






a. Work Performance 


12 factors 


no speciHc behaviors defined 


b. Worlc Behaviors 


two separate classes of behavior 
identiHed 


5 work behaviors listed 


c. Recording System 


rated on 5 point sctde 


none 


d. Frequency of Observation 


not mentioned 


frequent observations expected 


8. Reporting 






a. Formo 


standard forms for tdl phases 


standard forms for all phases 


b. Final Report Format 


optional computer generated report 


3 separate forms used to report 
different results 


9. Utility 






a. Vocational Exploration 


extensive information given to client 


some direct client use 


b. Vocational Recommendrtions 


future educational goals 


related to worker trait groups 


c. Oounsel Utilisation 


selor 




10. Training in the System 






a. Trzuning Required 


no 


no 


b. Training Available 


yes 


yes 


c. Duration 


4-6 hours 


2 days 




MVIk ll M r\1 A 




11. Technical Ccnsiderations 






a. Norm Base 


no data available 


19 different norm groups 


b, Reliibility 


no data available 


adequate data in manuals 
.74-97 


c. Validity 


no data available 


construct and concurrent reported 
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JEVS 


PREP 


Va'ipar 17 


client uses time clock 

from end of initructions to tuk 
completion 

v^Aed on 3 point scale 

random check, compared to standards 

minimal use 

most rated on S point scale 

time and quality given equal weight 


not relevant 
entire work sample 

no separate time norms 
compared to standards 
some use 

ratings based on industrial standards 
quzdity 


only one time score used in system 
preset for oim task 

used only for one task 
number of correct responses 
not used 

number of total points 
number of correct responses 


16 specific; 4 general factors speci- 
fied 

clearly defined 

3 point rating scale 
extensive observation 


no factors listed 

8 behaviors defined 

3 point rating system 
not specified 


no factors listed 

some specific areas defined 

3 point rating scale 
not specified 


standard forms for all phases 

narrative summary; standardised 
format 


standard forms for tdl phases 

narrative summary with work sample 
results 


standard forms for recording Sc 
scoring 

not used, depends upon facility 


limited 

highly related to DOT 
oriented toward counselor 


e;:ten8ive information given to client 

specific jobs Sc training related to 
DOT 

report zumed at counselor ii client 


some direct client use 
largely dependent upon user 

results of each task derigned for 
counselor usage 


yes 
yes 

i week 

not avzulable 


no 
yes 
1 day 

as requested by user 


no 
yes 

1 day or more 

as requested by user 


i,iuu Clients 

no data available 

no recent data are available 


industrial quality standards 
no data available 

content validation only 


"research norms" 
no data available 

no data available 
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SAM 


SAGE 


1. Development 

b. Target Group 

c. B&Bii of Syttem 


Pinfiv Mountain Pr^M 

mildly handicapped) disadvantaged 
students 

D-Map. GOE. GATE 


Train-Ease Corp 

students, dioadvantag<^d, some disabled 
DOT, GOE 


2, Organisation 

a. Number of Work Samples 

b. Grouping of Work Samples 

c. Manutd 


S tests, 12 wc~* samples 

each is independent 

general manual, many details lacking 


15 tests, work samples Sc inventories 

worker trait group variables 

separate section of manual for each 
test; contains all details 


3. Physical Aspscts 

a. Packaging 

b. Durability 

c. Expendable Supplies 

d. Replacement 

e. Computer Requirements 


packaged into 1 kit 
fairly durable 
tests and forms 
from developer 

Apple, IBM, 1 disk drive, printer 


each packaged separately 

appears durable 

no consumables except forms 

from distributor 

IBM, 2 disk drives, 640K RAM 


4. Vocational Evaluation Process 

a. Preliminary Screening 

b. Sequence of Work Sample 
Administration 

d. Evaluation Setting 

e. Time to Complete Entire 
System 


not required 

tests given first, work samples in any 
order 

some 

classroom atmosphere 
1 1/2 to 2 1/2 hours 


not required 
any order 

little during testing 
formal testing 
4 hours 


5. Administration 

a. Procedures 

b. Method of Instruction Giving 

c. Separation of Learning/ 
Performance 

d. Providing Assistance to 
Client 

e. Repeating Work Samples 


lacks details 

oral St demonstration 

clear separation 

no assistance during performance 
phase 

encouraged 


specified in detail 

oral or self « administered 

clear separation 

no assistance after timing begins 
if invalid results suspected 
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TAP 


TOWER 


Vaipar 


Talent Assessment Program 

above TMR students disadvantaged 

"functional aptitudes" 


Vocational Rehabilitation Administra- 
tion 

physically and smotionally disabled 

job analysis of possible jobs for 
disabled 


Valpar International 

general population, injared workers 

trait-and-factor 


10 

each is independent 

general manual contains all system 
details 


93 

14 training manuals 

single manual; some details not 
provided • 


19 

each is independent 

separate manual for et h work sam- 
ple; contains most details 


each packaged separately 

very durable 

forms and printer paper 

locally or from developer 

Apple, TRS-.80, IBM, 1 disk drive, 
printer 


ICD does not sell hardware, each user 
constructs own; this section not 
relevant for TOWER 

not applicable 


each packaged separately 
very durable 

few expendable supplies; forms 
locally or from developer 
not applicable 


not mentioned 

first and last test specified; rest any 
order 

little during testing 
formal testing 
2 to 2 1/2 hours 


emphasised for planning purposes 
progressive within 14 areas 

not specified 

realistic work setting stressed 
3 weeks 


not required 
discretion of evaluator 

minimal 

classroom or work place 
estimate 1 hour per work sample 


specified in detail 

wide variety of methods used 

considerable separation 

assistance can be provided after 
timing begins 

if desired by client 


some spei^iHed in detail 
written and demonstration 

not specified 

not speciHed 

encouraged for upgrading 


specified in detail 

oral and demonstration; limited read- 
ing 

11 work samples have formal practice 
periods 

not specified 
evaluator's decision 
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OUTLINE 


SAM 


SAGE 


6. Scoring and Norms 

a. Timing 

b. Timing Interval 

c. Time Norms 


cvaluator times client 

from end of practice for speciHed 
time period 

percentiles MTM 


electric and electronic timing devices 
speciHed period of time for each task 

converted to 5 or 6 point scales 


d. Error Scoring 

e. Scoring Aids 

f. Quality Norms 

g. Emphasis in Scoring 


not clearly defmed 

some use 
percentiles 

time Si quality given equal weight 


not relevant; errors not recorded 
extensive use 

not relevant; no errors recorded 

only number of correct responses are 
recorded 


7. Observation of Clients 

a. Work Performance 

b. Work Behaviors 

c. Recording System 

d. Frequency of Observation 


no factors listed 
8 behaviors defined 
S poliit rating scale 
CO- -^tant 


no factors listed 

19 separate behaviors defined 

4 point rating scale 

not specified 


8. Reporting 

a. Forms 

b. Final Report Format 


one form for data entry 

profile related to training programs; 
computerised 


standard forms for all phases 
worker trait profile results 


9. Utility 

a. Vocational Exploration 

b. Vocational Recommendations 

c. Counsel Utilisation 


very limited 

treuning programs within school 
report aimed at counselor and student 


very limited 

licts DOT titles and codes 
dependent upon user 


10. Training in the System 

a. Training Required 

b. Training Available 

c. Duration 

d. Follow-up 


no 

yes 

2 hrs to 1 day 

technical assistance available 


no 
yes 

1 day 
available 


11. Technical Considerations 

a. Norm Base 

b. Reliability 

c. Validity 


MTM 3 student norm groups 
test-retest over .80 
no date available 


4 different norm groups 

test-retest; KR>20 reasonably high 

other tests and ratings acceptable 
levels 
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TAP 


TOWER 


Valpar 


evaluator times client 

from end of instructions to task 
completion 

percentiles 


evaluator times client 

from end of instructions to task 
completion 

rated on 5 point scale 


evaluator times client 

from end of instructions to task 
completion 

actual time recorded; converted to 
MTM and percentiles 


penalty times added to time scores 
not used 

no separate quality scores 
time 


compared to standards 

extensive use 

rated on 5 point scale 

time Si quality given equal weight 


scored separately and combined with 
time score 

some use 

separate norms, percentiles 
MTM standards 


no factors defined; some listed 
no factors defined; some listed 
not used 
not specified 


only one factor defined 
a few listed in final report 
5 point rating scale 
not specified 


no factors defined 

17 work characteristics defined 

5 point rating scale 

not specified 


standard form 

computer generated format, profiles^ 
scores 


standard form for each phase 

narrative report using standard 
outline and rating 


separate form for each work sample 
none used, independent work samples 


very limit' i 

specific job titles listed 
oriented toward counselor 


direct client use - exposure to sev- 
eral areas 

limited to jobs related to work areas 
oriented toward counselor 


limited 

depends on use in facility 
cannot be specified 


yes 
yes 

1 1/2 days 
as needed 


yes 
yes 

1 and 2 week classes 
not available 


no 
yes 

depends on evaluator needs 
consultation basis 


7 different norm groups 
coefficient of stability .85 
no data available 


clients 

no data available 
equivocal results 


MTM; 11 norm groups 

'^ata available cannot be assessed 

no data available 
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OUTLINE 


VES 


VIEWS 


1. Development 






a. Sponsor 


Singer Educational Division 


JEVS 


b. Tftrgct Group 


special needs population 


moderately ir severely mentally 
retarded 


c. B&sis of System 


groups of tasks on related jobs 


DOT - worker ^k.ll groups 


2. OrgantEation 






a Niimhpr of Work Samoles 


28 


16 


b. Grouping of Work Samples 


e&ch is independent 


4 worker skill groups 


c. Manual 


4 basic manuals, detailed 

■ ■ 


single manual; very detailed 








a. Packaging 


each self-contained in carrel 


most individually in portable plastic 
cabinets 


b. Durability 


expect some problems 


very durable 




wood, metalf wire, chemicals, forms 


paper, f .ring, fiberboard, forms 




locally or from developer 


from developer 


6. Computer Re^juirements 


not applicable 


not applicable 








A PreliminMv Screenintr 


not required 


not required 


b. Sequence of Work Sample 
Administration 


discretion of evaluator 


from least to most complex 


c r^lipnt Tnvnivpmpnt 

w> X/<lCil« Ail V l^iV ClilCilv 


considerable client involvement 


extensive client involvement 


Q. ZfYaiuawion ^ewwing 




realistic work atmosphere stressed 


e. Time to Complete Entire 
System 


average 2 l/2 per work sample 


20 to 35 hours 


5. Administration 






a. Procedures 


specified in detail 


specified in detail 


b. Method of Instruction Giving 


audio-visual 


oral & modeling, flexibility in tech' 
nique stressed 


c. Separation of Learning/ 
Performance 


minimal 


Almost total; well established criteria 


d. Providing Assistance to 
Client 


checkpoints bulletin 


little assistance after timing begins 


e. Repeafr»ng Work Samples 


at request of client 


repeated if considered necessary 
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VITAS 


WREST 


WSD 


Manpower Administration 


Jastak Associates 


— 

Attainment Company 


educationally & culturally disad* 
vantaged 


severely disabled-mentally Sc physi- 
cally 


severely mentally retarded, mentally 
ill, & physically disabled 


GOE 


not speciHed 


3 basic prevccational skills 


21 


10 


20 


16 GOE Work Groups 


each work sample is independent 


function and difficulty 


single manual very detailed 


well-organized manual; contains all 
details 


single manual; most system details 
given 


each packaged separately 


system packaged in cardboard cartons 


■ ■■■ " 

each work sample packaged individu- 
ally 


durable 


durable 


durable 


paper, string, sheet metal, forms 


mostly paper products, forms 


plastic bags and forms 


from developer 


from developer 


extras provided, also order from 
developer 


not applicable 


not applicable 


not applicable 


not required 


not required 


not specified 


progressive from least to most com- 
plex 


discretion of evaluator 


by difficulty of task 


considerable client involvement 


some 


not specified 


realistic work setting stressed 


formal testing 


classroom or work activity center 


15 hours 


1 1/2 hours 


not relevant - training stressed 


speclHed in detail 


specified in detail 


■ 

specified in detail 


oral and demonstration 


oral and demonstration 


modeling and oral flexibility urged 


no separation 


considerable 


minimal 


little assistance provided 


no assistance after timing begins 


discretion of evaluator 


not recommended 


encouraged for upgrading 


repeat >d for upgrading 
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6. Scoring &nd Norms 

St. Timing 

b. Timing Interval 

c. Time Norms 

d. Error Scoring 

e. Scoring Aids 

f. Quality Norms 

g. Emphasis in Scoring 



7. Observation of Clients 

a. Work Performance 

b. Work Behaviors 

c. Recording System 

d. Frequency of Observation 



8. Reporting 

a. Forms 

b. Final Report Format 



VES 



evaluator times client 
varies with each work sample 

based on number of minutes to 
completion 

compared to criteria 

soma usQ 

5 point 8cal2 or subtracted from time 
score 

time & quality ^ven equal weight 



20 factors defined 
none listed 

not used-actual observation recorded 
not specified 



standard form for all phases 



no format given; includes description 
of contents 
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evaluator times client 

after task is learned to completion 

rated on 3 point scale; MODAPTS 

compared to standards 
some use 

rated on 3 point scale 

time Sc quality given equal weight 



10 factors defined 

clearly defined 

specific behaviors recorded 

constant 



standard forms for all phases 



standard format with behavior data Si 
work group 



0. Utility 

a. Vocational Ebcploration 

b. Vocational Reconunendations 

c. Counsel Utilisaticn 



extensive information given to client 
dependent upon user 

dependent on user 



little use to client 

related to DOT & Worker Skill 
Groups 

oriented toward counselor 
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10. Training in the System 

a. Training Required 

b. Training Available 

c. Duration 

d. Follow-up 



yes 

2 day, 1 or 2 weeks available 
available 



yes 

yes 

1 week 

not available 



11. Technical Considerations 

a. Norm Base 

b. Reliability 

c. Validity 



clients, employed workers, MTM 

test-ntest .61 or .71 
mostly content 



452 mentally retarded persons, 
MODAPTS 

no data available 

no data available 
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VITAS 


WREST 


WSD 


evaluator times client 


evaluator times client 


exact procedures not speciHed 


from end of instructions to comple- 
tion 


from end of instructions for specified 
period of time 


not speciHed 


rated on 3 point scale 


time to completion compared to scale 
scores 


some MTM norms 


compared to standards 


compared to standards 


compared to standards 


extensive use 


not used 


not used 


rated on 3 point scale 


all errors totaled for a single quality 
score 


percentage of the errors recorded 


time & quality given equal weight 


time 


time & accuracy given equal weight 


9 factors defined 


no fturtors defmed 


separate rating form 


several general factors deHned 


IC defined in general terms 


mostly on-task oH'^task 


specific behaviors reported 


scale from 1 to 18 


interval & event recording used 


almost constant observation stressed 


not specified 


carefully specified 


standard forms for all phases 


standard forms for recording perfor- 
mance 


standard forms for recording perfor- 
mance 


standard format, emphasis on work 
group 


numerous examples given in manual 


not mentioned in manual 


little use to client 


limited use to client 




related to DOT & work groups 
oriented toward counselor 


not specified 
not specified 


WSD is designed to teach very basfc 
discrimination, assembly & packaging 
skills 


yes 


no 


yes 


yes 


none 


yes 


1 week 


not applicable 


1 day 


not available 


not applicable 


included in purchase price 


secondary school students 


3 major groups, characteristics well 
defined 


MTM 


no data available 


test-retest .80's and .90's 


no data available 


no data available 


correlations between scores & super- 
visor's ratings .86 & .92 


no data available 
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OUTLINE 


WOWI 




1. Development 

a. Sponsor 

b. Target Group 

c. Basis of System 


Robert E. Ripley 

wide range: students, adults, business, 
VR clients 

DOT & GOE 




2. Organisation 

n. Number of Work Samples 

b. Grouping of Work Samples 

c. Manutd 


4 major testing areas 

4 vocational areas 

single manual, some details missing 




3. Physical Aspects 

a. Packaging 

b. Durability 

c. Expendable Supplies 

d. Replacement 

e. Computer Requirements 


not relevant 

reusable test booklets 

printer paper, test forms 

disk returned to developer for reset- 
ting 

IBM, I disk drive, 128K RAM 




4. Vocational Evaluation Process 

a. Preliminary Screening 

b. Sequence of Work Sample 
Administration 

c. Client Involvement 

d. Evaluation Setting 

e. Time to Complete Entire 
System 


not specified 

by vocational area 

considerable for a formal testing 
situation 

formal testing situation 
45 minutes to 1 X/2 hours 




5. Administration 

a. Procedures 

b. Method of Instruction Giving 

c. Separation of Learning/ 
Performance 

d. Providing Assistance to 
Client 

e. Repeating Work S^imples 


not given in manual 

computer administered, self-admin- 
istered 

minimal 

no procedures gi/en in manual 
not relevant 






OUTLINE 


WOWI 




6. Scoring and Norma 

a. Timing 

b. Timing Interval 

c. Time Norms 

d. Error Scoring 

e. Scoring Aids 

f. Quality Norms 

g. Emphasis in Scoring 


Not relevant; computer scored and 
printed either locally or sent to 
developer for scoring 




7. Observation of Clients 

a. Work Performance 

b. Work Behaviors 

c. Recording System 

d. Frequency of Observation 


no client observations made 




8. Reporting 

a. Forms 

b. Final Report Format 


separate answer sheet 

2 report formats, computer generated 




Q TTlrililrv 

a. Vocational Exploration 

b. Vocational Recommendations 

c. Counsel Utilisation 


little use to client 

no speciHc reconmiendations made 

as initial screening device 




10. Training in the System 

a. Training Required 

b. Training Available 

c. Duration 

d. Follow-up 


no 

yes-cassette tape 

not mentioned 

no information available 




a. Norm Base 

b. Reliability 

c. Validity 


several groups; procedures not given 
variety of methods most in .80 *s 
correlations with other tests 
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Apticom 



1. Development 

a. Sponsor - The Apticom was developed by the Vocational Research Institute 
division of the Philadelphia Jewish Employment and Vocational Services 
(JEVS). Unlike other JEVS products, this is not a work sample system in the 
strict sense of the word; it is a computerized assessment or testing system. 

b. Target Group - The Apticom is designed for English or Spanish speaking 
disadvantaged job applicants, high school or special education students, and 
rehabilitation clients. The apparent purpose of the system is to provide a 
quick vocational assessment of three major areas: aptitudes, interests and 
educational levels. The Apticom tests appear most useful for initial assess- 
ment. The purpose of the system is to combine aptitude, interest and aca- 
demic skills into meaningful job recommendations. 

c. Basis of the System - The aptitude part of the system is based on the U.S. 
Department of Labor definitions of 10 of the 11 Aptitudes (except Color 
Discrimination), with much of the design and test item f .mats closely resem- 
bling those of the General Aptitude Test Battery. The Occupational Interest 
Inventory is based on the twelve interest areas (e.g.. Artistic, Selling) defined 
in the Guide for Occupational Exploration . This interest measure was devel- 
oped from the USES Interest Inventory. The Educational Skills Development 
Battery was derived from the General Educational Development (GED) Lan- 
guage and Mathematics scales; there are six levels of these two scales. In 
summary, each part of the system is based on specific DOL job analysis and 
test variables. The results cross-match the GATE Occupational Aptitude 
Patterns (GAPs) (U.S. Department of Labor, 1980) with the GOE Work Groups 
(i.e., the 66 four digit GOE codes). 

2. Organization 

a. Name and Number of Work Samples - The Aptitude section contains 11 
specific tests measuring ten aptitudes: G, V, N, S, P, Q, K, F, M and E:^ 

(1) Object Identification - P (1 minute; 30 items) 

(2) Abstract Shape Matching - P (2 minutes; 30 items) 

(3) Clerical Matching - Q (1 minute; 30 items) 

(4) Eye-Hand-Foot Coordination - E (45 seconds) 

(5) Pattern Visualization - S and G (5 minutes; 3C items) 

(6) Computation - N (4 minutes; 30 items) 

(7) Finger Dexterity - F (2 minutes) 

(8) Numerical Reasoning - N and G (7 minutes; 23 problems) 

(9) Manual Dexterity - M (2 minutes) 

<IG) Word Meanings - V and G (2 minutes, 15 seconds; 30 items) 

(11) Eye-Hand Coordination - E (45 seconds) 

The Occupational Interest Inventory is a single test of 162 items, using a 
like-?-dislike format. 



2 See Glossary for definitions. 

31 




36 



The Educational Skills Development Battery contains two separate tests: 

(1) Language Skills Development (10 minutes; 30 items) 

(2) Math Skills Development (15 minutes; 30 items) 

b. Grouping of Work Samples - The tests are grouped into the three areas listed 
above. Individual tests within each area are administered in the order listed 
above. 

c. Manual - The system contains an administrative and a technical manual. The 
administration manual begins with very detailed directions and diagrams on 
setting up the administration and scoring equipment. There follows a separate 
section for each test battery and for each test within each battery. All 
instructions to be read verbatim are contained in "shadow boxes"; the accom- 
panying demonstrations are also printed. The administration manual also 
contains detailed procedures for both hand and machine scoring, various 
conversion charts and tables. The format promotes easy use and the content 
of the manual is complete. 

The technical manual consists of development and interpretation sections for 
each of the three parts of the battery. These sections contain explanations 
for all development steps that are easy to follow and understand. 

Physical Aspects 

a. Packaging of Work Samples - The Apticom system is completely self-con- 
tained. The entire system is sold as a unit. The Apticom consists of a 
dedicated computer and several testing devices. The major device is a plastic 
board about 18 by 24 inches, containing an array of holes that correspond to 
test answers. Except for the apparatus tests, all test items are contained on 
plastic overlays that are placed on this board. The client uses a wand to 
select between alternatives. The system also includes various computer con- 
trolled devices for measuring the dexterity aptitudes. 

b. Durability - The computer hardware is sturdy and portable, but should be 
handled with care in transit. Test overlays can be cleaned with mild soap 
and water. This reviewer recommends careful use and storage of this equip- 
ment. 

c. Expendable Supplies - The only expendable supplies are the various report 
forms and paper for the printer (optional). 

d. Replacement - All replacement parts must be obtained from VRI. If hand 
scoring is used, the school or facility may duplicate the appropriate forms for 
local use. It must be pointed out that unlike many work sample systems, the 
Apticom is a computer driven test system requiring no supplies such as wood, 
metal or plastic. 

e. Computer Requirements - The heart of the system is a dedicated computer 
that is critical for all administration and scoring. Because the Apticom is 
equipped with an RS-232-C serial interface, a serial printer can be connected 
directly to the control console. The results can also be downloaded to an 
IBM-i C or IBM compatible computer for storing client results. 
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4. Vocational Evaluation Process 



a. Preliminary Screening - No mention of preliminary screening is given in the 
manual. However, this reviewer believes that the Apticom is best used for 
this function. 

b. Sequence of Work Sample Administration - Although there is no set sequence 
for administering each of the three batteries, the tests within each battery 
are administered in a set sequence. The aptitude tests are administered in 
the order they are listed in section 2.a. This arrangement is intended to 
alternate between the dexterity, perceptual and cognitive aptitudes to reduce 
fatigue. In the Educational Skills Development Battery the Language skills 
arc administered first. 

c. Client Involvement - There appears to be very little client involvement during 
the actual testing process. Although the manual contains no provision for 
sharing the results with the student or client, the Apticom report format is 
designed for client use. 

d. Evaluation Setting - Although not specifically stated in the manual, the 
Apticom creates a formal testing atmosphere. 

e. Time to Complete the Entire System - The timed sections of the Aptitude 
Test Battery total 28 minutes, 15 seconds. The timed portions of the Educa- 
tional Skills Development Battery total 25 minutes. These times do not 
include instructional and practice times. The Occupational Interest Inventory 
is untimed; most persons should be able to complete it in about 20 minutes. 
The entire Apticom could be administered in about i and 1/2 to 2 hours. 

5. Administration 

a. Procedures - Each test is clearly explained and information needed for correct 
adminstration is provided. 

t. Method of Instruction Giving - All instructions are oral, accompanied by some 
demonstration. The e^'aluator reads the instructions directly from the ad- 
ministration manual. Although reading is required for many of the tests, 
none of the instructions require reading. The only in;..-uments that include 
items beyond a fourth grade reading level are power tests specifically de- 
signed to assess verbal aptitude and language skills. 

c. Separation of Learning/Performance - The student is given a very short 
practice session prior to each test. There are no specific criteria io be 
reached in order to move from the practice section to the performance sec- 
tion. Th s, although there is some separation of learning and performance, it 
is entirely possible that some clients would start the tests without clearly 
understanding the instructions. The highly speeded practice and test sessions 
only increase this problem. 

d. Providing Assistance to the Client - Apparently assistance is allowed during 
demonstration and practice; the manual does not give any procedures for 
helping clients having difficulty once the performance session has started. 
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c. Repeating Work Samples - Thii *s not mentioned in the manual. Because of 
the practice effect it is assumed that the tests are not to be repeated. 

Scoring and Norms 

a. Timing - The computer contains pre-set times for each test, except the 
interest inventory. Timing for each test is started by the evaluator. Hc/Shc 
simply presses the start button. The system will beep and turn the control 
lights off when the predetermined time is up for each test. 

b. Timing Interval - The teacher/evaluator begins timing after the completion of 
the practice session. 

c. Time Norms - This is not relevant for the Apticom. 

d. Error Scoring - The final raw score for each test, except the interest inven- 
tory, is the number of correct responses. 

e. Scoring Aids - The entire Apticom is completely computer scored. 

f. Quality Norms - This is not relevant for the Apticom. 

g. Emphasis in Scoring - This is not relevant for the Apticom; the final raw 
score is the number of correct responses for each test. 

There are two basic ways to score the Apticom results. In the manual pro- 
cedure, the raw test scores are read from a display following each test. These 
scores are recorded on various forms (see below) and then converted using a 
series of norms tables. If a printer is available, the dedicated computer 
performs all scoring and the final report is complete with explanations of all 
test variables that are printed. 



Observatio n of Clients 

a. Work Performance - 

b. Work Behaviors - 

c. Rating System - 

d. Frequency of Observation - 
Reporting 



There is no mention of behavioral 
observations in the manual. Because 
of the short testing time and the 
formal nature of the test, client 
observation is not relevant. 



a. Forms - There are three Apticom profile sheets used in hand scoring: 

- The Individual Aptitude Profile consists of data and client identification 
information, followed by a chart that plots the 11 tests to measure the ten 
aptitudes. It is possible to chart a grade adjustment for ninth and icnth 
grade students. As with the GATE, the standard error of the measurement 
(SEm) can be added to each aptitude score. A percentile equivalent can 
also be calculated. 
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- The Occupational Interest Inventory Profile Sheet consists of two charts 
for plotting the 12 GOE interest areas. Interest inventory scoring is based 
on the frequency of "like" responses. 

- The Educational Skills Development Profile Sheet contains client identifica- 
tion information and scores on each of the first four GED scales. 

b. Final Report Format - There are several options for producing a final report 
cither by downloading the results on a PC or by directly printing from the 
Apticom computer. The contents of the standard report are as follows: The 
Aptitude Test Battery section contains the raw and standard test scores for 
each of the 11 tests. Aptitudes are given in standard scores and percentiles 
as well as graphically. The Occupational Interest Inventory gives the num- 
ber of -like/ and Mislike" scores for each of the 12 interest areas and 
the percentile for each score, compared to the entire norm group and to male 
and female subgroups. Standard scores and a graphic display are presented 
for each interest scale. A statistical idiographic analysis is also conducted to 
establish the client's high interest <\rea(s). The Educational Skills Develop- 
ment Battery section contains the number of correct responses for each of 
the four GED levels and a content analysis. 

The iicart of the final report is the Vocational Recommendations section. 
This section lists all four-digit GOE codes (i.e., Work Groups) that are viable 
based upon the client's high interest areas and aptitude scores (when the 
latter are compared against GATE Occupational Aptitude Patterns). The 
report also includes examples of specific jobs, with DOT codes and GED and 
SVP requirements, based upon the GED scales attained on the mathematics 
and language achievement tests. The recommendation section then provides 
cross-classification of interest, aptitude and educational levels. 

9. Utility 

a. Vocational Exploration - The Apticom does not provide any actual experiences 
that reflect competitive employment. It is strictly a testing system. 

b. Vocational Recommendations - The Apticom provides Work Group and job 
matches. These are specific and state the DOT title and code number. The 
specific vocational recommendations are the central feature of the system. 

c. Counselor Utilization - The computer generated report would be very useful 
for providing specific information to the referring counselor or teacher. 

10. Training in the System 

Training Required - No training is required prior to purchase. 

Training Available - The availability of training is not mentioned in the 
manuals. More recently, however, VRI has introduced a 40 r;inute VHS 
videotape with workbook that reviews system set-up, operation, administration 
and score/report interpretation. 

Duration - Not mentioned in manual. 
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a. 
b. 



ERIC 



d. Follow-up - Not mentioned in manual. A toll-free telephone number is pro- 
vided for customer use. 

1 1. Technical Considerations 

a. Norm Base - The aptitude battery was developed on selected samples of 
younger persons in the U.S. and v/as standardized on adults in the U.S. and 
Canada. The interest inventory contains two separate norms: (I) secondary 
students and (2) employed adults. Because the educational battery was devel- 
oped using a criterion referenced approach, norming was not necessary (Hupp 
and Donfrio, 1983). In general all samples are fully described and most arc 
of adequate size. The development of norms was straightforward and tech- 
nically sound. 

b. Reliability - Internal consistency, standard error of the measurement (SEm) 
and test-retcst reliabilitj coefficients are presented for both tests and ap- 
titudes. Aptitude test-revest reliabilities range between .65 and .89, with most 
being higher than .80. Although these are adequate, the SEm's are rather 
large when compared to the frequency distributions. Alpha coefficients and 
test-retest reliabilities were presented for the 12 interest areas. These 
results were compared to the USES Interest Inventoiy. The shorter Apticom 
interest scales had in almost all cases just as high or higher reliability coef- 
ficients than the USESII. No reliability data were reported for the educa- 
tional skills tests. 

c. Validity - The aptitude tests and interest inventory were validated against 
their U.S. DOL counterparts. Correlations between Apticom and GATE ap- 
titudes were in the .80's for the three cognitive aptitudes, in the .60's for 
the perceptual aptitudes, and the .50's for dexterity skills. The manual also 
compares factor analyses of the GATE and Apticom aptitudes. The GATE 
factor structure resulted in two factors, interpreted as cognitive/perceptual 
and motor. The Apticom has three factors: cognitive, perceptual and motor. 
The Apticom developers rightly interpreted this as a strong indication of the 
soundness of the Apticom aptitude tests. The interest inventory was validated 
against its USES equivalent. Correlations reported for three samples ranged 
form .67 to .90. When the length of the Apticom interest inventory is taken 
in account, these are quite high. The Educational Skills Development Battery 
wa$ developed using a content validity approach in which all test items were 
rated by experts for their GED level.^ 

Thus, at present all reported validity is based on correlations with widely 
used USES tests. Although the reported results are very impressive, the 
Apticom definitely needs empirical validity. 

12. Reviewer's Summary and Contents 

The Apticom is designed to provide either a quick assessment of a person's voca- 
tional capabilities or as an initial screening device for a longer vocational cvalua- 
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^ There are six levels of Mathematical and Language development. The Apticom 
was developed using only the lower four levels. These four levels range from grade 
one to grade 12. Given the target population of the Apticom, these four levels appear 
to be appropriate. 
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tion. The central part of the system is its ability to combine the three factors 
commonly used in vocational testing, i.e., aptitudes, interests and academic per- 
formance levels. Of these the relationship between aptitudes and interests is the 
most critical. This is not a new concept nor arc the basic types of tests original. 
According to its manual, the major reason for the existence of the Apticom is to 
reduce client/student test anxiety or boredom. In addition, the total testing time 
to obtain these data is much shorter than it would be if the corresponding USES 
tests were used. 

If the information reported in the technical manual is correct, the initial develop- 
ment of the Apticom has resulted in a technically sound test battery. The re- 
markable thing about this development is that the highly speeded Apticom tests 
have reliability coefficients that meet or exceed those of the much longer USES 
tests. The VRI staff can be very proud of this. As stated above, the initial 
validation of the instrument against USES tests has resulted in significant con- 
struct validity coefficients. Although this is an excellent beginning, the instru- 
ment defiritely needs concurrent and predictive validation research. 

The manuals are easy to understand and follow, the standard printed report 
contains all necessary information and the physical apparatus is easy to operate 
and control. These features, plus the short administration time and the reported 
technical aspects, make the Apticom an impressive instrument. 

13. Address 

Vocational Research Institute 
21G0 Arch Street 
Philadelphia, PA 19103 

14. Cost 



Single Apticom Unit $5,350.00 

Single Apticom Unit with printer. $6,000.00 

Midi System (2 Apticom units, master control, 2 printers) $12,400.00 

Maxi System (4 Apticom units, master control, 4 printers) $22,300.00 

Spanish/Bilingual Kit (overlays/manual) $495.00 
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The Career Evaluation System 



(Series 100, 200/230 and 300) 
(CES) 



Development 

a. Sponsor - Originally developed by Goodwill Industries of Chicago under a 
Rehabilitation Services Administration, U.S. Denartment of Education grant, 
the CES was further developed, refined and r^arketed by Career Evaluation 
Systems, Inc. 

b. Target Group - The three major divisions of the Career Evaluation System are 
intended for different populations: The Series 100 tests a normal population 
from ages 16 to 65 for selection and training in business and industry. The 
Series 200, the original system, is intended for assessing a wide variety of 
persons with physical, sensory and/or psychological handicaps. A subset of 
the Series 200, the 230, tests either the general population or disabled per- 
sors who are low readers or non-readers. The Series 300 was designed for 
mentally retarded persons. The systems have been used successfully in 
schools, pain centers, hospitals, correction facilities, manpower training pro- 
grams, rehabilitation facilities and industry. 

c. Basis of the System - Originally based almost exclusively on the third edition 
of the Dictionary of Occupational Titles, the system's data base (see 11c.) has 
been revised for the newer fourth edition and the Supplement. The Series 
tests isolated traits (or specific aptitudes) and combines these into scores for 
each function on the Data-People-Things "hierarchy." for example, a client's 
optimum job level could be classified as a .231 (r.e.. Data level of 2, People 
level of 3, and Things level of 1). The initial selection of the isolated trait 
tests and factors were derived from research done by Dr. Hester in the late 
1960's and 1970's. Physical and environmental conditions. General Educational 
Development, and Specific Vocational Preparation are also considered. CES is 
not a work sample system; it is a battery of psychological tests ar 1 ratings 
designed to relate client scores to the DOT. 

Organization 

a. Name and Number of Work Samples - Each Series uses a different number of 
pencil-and-paper tests, apparatus tests and ratings. The name of the test, 
the specific aptitude(s) measured and the number of the Series it is in are as 
follows: 

(1) Minnesota Paper Form Board - Spatial perception; Series 100, 200/230 
and 300. 

(2) (Raven) Standard Progressive Matrices - Abstract reasoning; Series 100, 
200/230 and 300. 

(3) SRA Verbal Form - Verbal and numerical reasoning; Series 100 and 200. 

(4) Gates-MacGinitie Reading - Reading comprehension; Series 100, 200/230 
and 300. 

(5) Wide Range Achievement Test (Revised) - Arithmetic level; Series 100, 
200/230 and 300. 

(6) SRA Leadership Opinion - Leadership; Series 100 and 200. 
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SRA Sales Attitude - Sales/persuasion; Series 100 and 200. 

IPAT CAB-Cf (Hidden Designs) - Perceptual accuracy; Scries 100 and 

200/230. 

IPAT Similarities - Perceptual accuracy; Series 300. 

IPAT CAB-Cs - Decision speed; Series 100 and 200. 

Purdue Pegboard - Finger dexterity; Series 100, 200/230 and 300. 

Minnesota Rate of Manipulation - Manual dexterity; Series 100, 200/230 

and 300. 

Etch-a-Sketch with overlay - Two-hand coordination; Series 100, 
200/230 and 300. 

Electro-tach Tachistoscope - Perceptual speed; Series 200/230 and 300. 
Multi-Choice Reaction Test - Reaction time and response orientation; 
Series 200/230 and 300. 

PTI Oral Directions Test - Following direction; Series 100, 200/230 and 
300. 

Hand Dynamometer - Hand strength; Series 100, 200/230 and 300. 

Hole Steadiness Plate - Arm-hand steadiness; Series 100, 200/230 and 

300. 

Hole Steadiness Plate with backplate - Precision aiming; Series 100, 
200/230 and 300. 

Tapping Board - Wrist-finger speed; Series 100, 200/230 and 300. 
Two-Arm Tracing Test - Two-arm coordination; Series 100, 200/230 and 
300. 

Polar Pursuit Tracker - Fine perceptual motor coordination; Series 
200/230 and 300. 

Bennett Hand Tool Test - Hand-tool dexterity; Series 200/230 and 300. 

Foot-operated Stapler - Multi-limb coordination; Series 200/230 and 300. 

Paper Feeding Machine - Machine feeding; Series 200/230 and 300. 

Depth Perceptual Test - Depth perception; Series 200/230 and 300. 

Lifting Platform - Isometric lifting; Series 200/230 and 300. 

Mirror Tracing Apparatus - Visual motor reversal; Series 200/230 and 

300. 



In addition to these tests, the evaluator rates the client's ability on several 
People Functions (i.e.. Mentoring, Negotiating and Entertaining) in the Series 
100 and Series 200/230. 

Grouping of Work Samples - Although each test is treated as an independent 
variable in scoring, the tests are grouped into four categories for 
administration: pencil-and-paper tests (1 through 10), apparatus tc3ts (11 
through 17), tests using the master control unit (18 through 22) and non- 
mobile tests (23 through 28). In addition, the printout containing the results, 
classifies the tests into se\en groups or second order factors: unilateral 
motor, bilateral moto*", perceptual-motor coordination, perceptual, intelligence, 
achievement and People test functions. 

Manual - There is a separate manual for each series. Each contains the 
following chapters: Introduction, General Administration, Apparatus Tests, 
Paper/Pencil Tests, Data Entry Instructions, Interpretation of Printout and 
Appendices. Each manual gives detailed procedures for administration and 
scoring, samples of data entry forms, and computer printouts of results with 
interpretation. The manuals are well organized and very easy to use. 
Finally, there are short manuals for telecommunications and equipment set-up. 
There is also a training manual. 
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3. Physical Aspects 



a. Packaging of the Work Samples - Each apparatus and test are packaged 
separately for shipping. The standardized psychological tests can be ordered 
directly from their respective publishers. 

b. Durability - Although no specific information on repair and replacement rates 
v/ere available, the apparatus should be fairly durable if properly maintained. 
Some of the apparatus must be recalibrated regularly. 

c. Expendable Supplies - The only apparatus tests using expendable supplies are 
the paper feeding machine and the foot operated stapler. The CES uses many 
test forms, answer sheets, etc. 

d. Replacement - Because the systems use precision apparatus, all replacement 
parts must be ordered from the developer. Paper-and-pencil tests can be 
ordered from their publishers. 

e. Computer Requirements - When using a telecommunications system for scoring, 
the user first enters raw test scores into a microcomputer; these are trans- 
mitted to CES via a modem for scoring. The results are transmitted back to 
the user, written on the disk, final computations are completed, and printed 
on his/her printer. Telecommunications software is available for both IBM 
and Apple. The following specifications are for IBM-PC/XT or AT: at least 
128 K RAM and two disk drives, monitor, printer and Hayes 1200 baud Smart- 
Modem. Apple computers require: at least 128 K RAM, two disk drives, 
monitor, printer and Hayes 1200 baud SmartModem. In addition, the ap- 
propriate cards, DOS, and cables are required. 

4. Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. The systems 
can be used alone as assessment devices or as screening tools at the begin- 
ning of a longer vocational evaluation. 

b. Sequence of Work Sample Administration - With one exception, the tests do 
not ha>^e to be given in any specific order. The Series 100 manual contains 
suggested schedules for group testing. 

c. Client Involvement - Because of the formal nature of the testing process and 
the emphasis upon accurate measurement, there is little client involvement 
during actual testing. 

d. Evaluation Setting - The assessment process results in a formal testing at- 
mosphere. The emphasis on accurate measurement using psychophysical de- 
vices to determine reaction time, etc. creates a laboratory setting. 

e. Time to Complete the Entire System - Because the number of tests differs 
with each Series, each system takes a different amount of time. The Series 
100 tests can be given in about 3 1/2 hours, with up to 12 persons tested at 
one time. Series 200/230 is administered in about 4 hours; the paper-and- 
pencil tests can be given in small groups of 5 or six. It is estimated that 
the Series 300 can be given to small groups in 4 hours. In each Series the 
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apparatus tests are given individually; the pencil-and-paper tests are given in 
small groups. 

Administration 

a. Procedures - For each test the manuals give the purpose, materials, test 
conditions, practice period (if any), administration, scoring and data recording. 
In addition, test manuals arc provided for all commercially available tests. 
All procedures are thoroughly defined. 

b. Method of Instruction Giving - All instructions arc read aloud to the client 
and many are accompanied by short demonstrations. The communication 
method may be varied to accommodate special client or student problems (e.g., 
hearing loss). Reading is only required for those tests where reading is part 
of the testing process. 

c. Separation of Learning/Performance - The manual emphasizes this separation 
and urges that, within the limits of the test, the student should have a 
practice period to understand the test. 

d. Providing Assistance to the Client - The evaluator is to make certain that the 
testee fully understands the instructions for each test. No assistance is given 
during the actual administration of the tests. 

e. Repeating Work Samples - Tests may be readministered *f there is reason to 
believe that a student or client was erratic or he/she was not functioning at 
his/her normal level. Testing may be repeated within two or three days of 
initial administration. The higher score is usually reported. 

Scoring and Norms 

a. Timing - The evaluator times the client. The timing of the psychophysical 
tests is carefully controlled by electric timing devices. A stopwatch or other 
timing device is used for the paper-and-pencil and dexterity tests. 

Although all results are scored by CES in a central location, there are three 
methods of transmitting assessment results: (1) use of a personal compu^:r 
with a modem for direct scoring, (2) mailing in the Data Entry Form, and (3) 
telephoning in the results to CES clerical staff. 

b. Timing Interval - For many tests, timing is the speed with which the client 
responds to a specific stimulus by performing highly unique responses. A few 
tests are timed from cither start to completion or for a set period of time. 

c. Time Norms - No separate time norms are presented. 

d. Error Scoring - A few separate error scores are computed. The psychophysi- 
cal tests use mostly time to completion or the number of responses performed 
within a definite time limit. 

e. Scoring Aids - A few transparencies are used to quickly identify correct 
answers. 

f. Quality Norms - This is not applicable; no quality scores are used. 
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g. Emphasis in Scoring - The emphasis is on time to completion, number of 
responses performed within a definite time limit, or number of correct re- 
sponses. 

7. Observation of Clients 

a. Work Performance - No work performance factors are observed. 

b. Work Behaviors - The Series 300 lists 15 separate behaviors to be observed 
under three headings: job performance and productivity, work habits and 
attitudes, and social behaviors. No specific observation methods or proce- 
dures are given in the manual. 

c. Rating System - In the Series 100 and Series 200/230 the People functions of 
Mentoring, Negotiating and Entertaining^ are rated on a six point scale. 
Each scale value is generally defined. In the Series 300, work behaviors are 
rated on a five point scale, ranging from work activity center level to com- 
petitive employment. 

d. Frequency of Observation - Because no work performance or behaviors are 
directly observed, this is not relevant. 

8. Reporting 

a. Forms - Standard forms are used to record responses on most of the psycho- 
physical tests. All data, together with demographic information, are trans- 
ferred to the Data Entry Form prior to computer scoring and job selection. 
Although there are separate versions of the Data Entry Form for each Series, 
all forms contain: agency information, client demographics. People relations, 
test scores. In addition, the Series 200/230 lists physical and environmental 
limitations; the Series 300 contains the 15 work behaviors. 

b. Final Report Form - The same basic report format is used for each Series. 
"The computer transforms raw test scores into scale scores using a 1 to 6 
scale. The highest score attainable is 6.0 and the lowest is 1.0." The print- 
out contains four major sections: (1) the demographic and identification 
information supplied by the user, (2) the scores on each test, on a scale of 
between 1.0 and 6.0 (in the Series 200/230 and 300 these are also given 
graphically), (3) the Data-People-Things "hierarchies" showing the client level 
of functioning, and (4) specific DOT occupational titles and codes, the DOT 
page number of the occupational definition, GOE codes, strength requirement, 
GED and SVP. The jobs listed are classified into four groups, depending on 
the probability of occupational success for the client: very feasible, feasible, 
possible and conceivable. 



* In the Series 200/230 People functions of Speaking/Signaling, Serving and 
Following Directions can be determined either by rating or by scores on the Oral 
Directions Test. 
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9. 



Utility 



a. Vocational Exploration - The formal testing atmosphere and the lack of 
introductory explanations relating the tests to jobs offers the client almost no 
chance for vocational exploration. 

b. Vocational Recommendations - The major purpose of the system is to make 
specific vocational recommendations. As stated above, the printout lists 
specific job titles. 

c. Counselor Utilization - The system is designed to report jobs that are within 
the client's or student*s abilities. This information, if communicated to the 
referring counselor effectively, could be very useful as a realistic basis for 
client choice, to broaden student vocational expectations, to help screen local 
opportunities, and to plan training for technical vocations. 

10. Training in the System 

a. Training Required - Training and certification are required prior to use. 

b. Training Available - Training is given on audio-cassettes; there is also a 
certification test. 

c. Duration - Training is approximately 12 hours, at trainee's discretion. 

d. Follow-up - This is available by telephone. On-site, hands on training is 
available on request. 

1 1. Technical Considerations 

a. Norm Base - No information on norms is given in any of the manuals. How- 
ever, CES states they are in the process of developing normative data; this is 
based on cases that already exist in the system. 

b. Reliability - Test-retest reliabilities for individual tests on 45 clients retcsted 
after four weeks ranged from .72 to .95; these are high. The second type of 
reliability is the duplication of the list of DOT defined jobs. In a test-retest 
situation, 78% of the job families listed on the first printout were the same 
as those listed on the second. 

c. Validity - There are very little data. The construct validity of the system is 
based on several factor analyses; however, none of these are given in the 
manuals. A single concurrent validation study of 156 dentists demonstrated 
that 80% of the dentists "would have been recommended to enter dentistry." 

A few words must be said about data bases. The Series 200/230 has a data 
base of over 2,400 occupations carefully selected from the DOT; the Series 
100 has over 2,300. There is considerable overlap oetween the two. The 
Series 300 data base contains over 1,400 separate jobs. The jobs in each data 
base were selected on the basis of: (1) commonly available in the national 
economy ana (2) representative of a group of wloscly-related occupations. 
The Series 300 data used a third criteria: the jobs were wifhin the com- 
petencies of mentally retarded persons. 
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Reviewer's Summary and Comments 



The Career Evaluation Systems Scries 100, 200/230 and 300 use the human factors 
approach that has been a test development model for over 40 years. This ap- 
proach has proven successful for many psychological tests. Each Series attempts 
to present a picture of the client's or student's specific aptitudes and to match 
these aptitudes with the structure of the DOT. This logical structure has a 
definite appeal to persons who stress aptitude testing either as part of a longer 
vocational evaluation process or depend on this testing for assessment. 

Career Evaluation Systems, Inc. has used a unique approach in taking one testing 
system originally designed for all disabled persons and dividing it in three separ- 
ate systems aimed at different populations. This results in increased flexibility 
for the teacher or evaluator. The sysiem does not claim to be a complete voca- 
tional evaluation system; CES realizes the need for occupational information, 
interest testing and evaluator interaction with the client. The three Series could 
be best described as a very logical series of tests designed to relate client abili- 
ties to the Data-People-Things "hierarchies" of the DOT. The chief advantages of 
the three Series are: (1) the ability to test a large number of persons in a short 
tinnie, (2) the direct relation between testing results and jobs in the DOT, (3) the 
use of what the developer calls a "fuzzy logic* approach to sort selected jobs in 
four gioups by predicted probabilities of success, and (4) the inclusion of specific 
tests to measure at least some of the People functions. 

The problems with the three CES Series are largely technical. There arc no 
technical manuals or technical sections within the manuals containing normative 
information, reliabilities and validation studies. Although many of the standard- 
ized tests (e.g.. Wide Range Achi'ivement Tcac) have adequate technical data in 
their respective manual the CES developed tests do not have comparable data. 

Address 

Career Evaluation Systems, Inc. 
/' ^8 Milwaukee Avenue 
Niles, Illinois 60648 



License Fee (This one time fee includes start-up costs, training cassettes, system 
manual, test manuals, scoring keys, data entry forms, and newsletter subscription) 



Package B (instruments used with master control unit foi automatic scoring) 
32,090.00 



$2,500.00 



Package A: Telecommunication Software 
For Series 100, 200/230 and 300. .. . 

Series 100 

Series 200/300 

Series 300 



$1,990.00 
$650.00 
S950.00 
S650.00 



Package C (apparatus tests) . 



$2,050.00 



Package D (paper-and-pencil tests) 



$370.00 
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optional non-mobile test apparatus (total). 



$2,835.00 



Complete Systems without Package A: 

Series 100 

Series 200/230 

Series 300 



$6,800.75 
$6,985.25 
$6,935.90 



Note: Individual components of Packages B, C, and D can be ordered. Thus, if 
the facility already has some of the testing equipment, they need not re-purchase 
it. 
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Key Educational Vocational Assessment System 



(KEVAS) 



1. Development 

a. Sponsor - The system was developed by Dr. Alvin Krass of Key Education, 
Inc. of Shrewsbury, New Jersey. 

b. Target Group - KEVAS is designed for a wide variety of both handicapped 
and "normal" populations, including high school students, dislocated workers 
and competitively employed adults. Because the language based measures can 
be varied to fit the assumed reading skills of the group tested, the system 
can serve a wide range of abilities. 

c. Basis of the System - According to the manual, KEVAS was designed "to tap 
basic elements of psychophysical functioning, that is, those underlying percep- 
tual skills which are fundamental for learning." The system contains a wide 
variety of cognitive, perceptual, dexterity and attitude tests and measures.^ 

"KEVAS development is focused toward integrating the vccaCional matching 
protocol, which matches individual performance profile to occupational and/cr 
training requirements. To date, functional criteria have been developed ^or 
more than 600 occupations and 75 vocational training programs" (Krass and 
Conlon, 1986, p. 225). 

2. Organization 

a. Name and Number of Work Samples - The system is a combination of ap- 
paratus, achievement, interest and personality tests. Some were developed by 
Key Education Inc.; others were commercially available tests. The tests and 
what they purport to measure are as follows: 

(1) Key Audiometer - Hearing. 

(2) Key Auciom-C'cer - Auditory localization. 

(3) Key Audiometer - Auditory memory. 

(4) Ciba-Geigy Chart - Visual acuity. 

(5) Ciba-Geigy Chart - Color acuity. 

(6) Key Audiometer and Key Tabletop Lab/Learning Code Component- 
Visual memory. 

(7) Key Fine Motor Skills Test - Manual dexterity. 

(8) Key Tabletop Lab/Reaction Time Component - Auditory reaction time. 

(9) Key Tabletop Lab/Reaction Time Component - Visual reaction time. 

(10) Key - What's Next? - Abstract reasoning ability. 

(11) Key Tabletop Lab/Learning Code Component - Non-verbal learning abil- 
ity. 

(12) Key Tabletop Lab/Learning Code Component - Response to stress. 

(13) Key Tabletop Lab/Dynamometer Component - Persistence. 

(14) Key Form S-3 - Literacy skills. 



^ The reader should note that unlike most work sample systems, KEVAS has no 
relationship to the DOT or other job classification systems. 
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(15) Key >^rithmetic Test - Arithmetic skills. 

(16) Wide Range Achievement Test Reading Scale - Reading level. 

(17) Key Vocational Interest Inventory and Self-Directed Search - Vocational 
interest. 

(18) Key Social Competence Rating - Social competency. 

(19) How Supervise - Supervisory potential. 

(20) Sentence Completion & Graphic Projectives - Personality attributes. 

b. Grouping of Work Samples - Because many of the measures use the same 
piece of apparatus, several assessments are made at the same time. All tests 
are administered in a prescribed order. The manual classifies the tests into 
three general groups: (1) measures of psychophysical functioning (e.g.. Fine 
Motor Skills, Color Acuity), (2) measures of work-related competencies (e.g.. 
Literacy, Spatial Relations) and (3) attitudinal and motivational measures (e.g.. 
Social Competency, Supervisory Knowledge). 

c. Manual - The system manual, contained in a single loose-leaf binder, gives 
the procedures for test administration plus supplementary material on the 
KEVAS. The manual contains the following sections: Historical Perspective 
of Test Development, Description of the Key System, Administration of the 
Key System, Key System Output and General Information. The administration 
section contains complete instructions for administering all tests and record- 
ing the results on scoring sheets. In the few places where ratings or judg- 
ments need to be made, rating criteria are clearly defined. Because all 
measurements are scored by Key Education, Inc. no scoring procedures are 
given in the manual. No information on how to interpret results is included. 
Finally, there is no information of sharing results with clients. 

Physical Aspects 

a. Packaging of the Work Samples - The apparatus is self-contained. The paper- 
and-pencil measures arc packaged in folders. Examiner recording sheets, 
literacy measures and checklists are packaged in predetermined quantities. 
All test materials arc shipped in sturdy, portable boxes. 

b. Durability - Most of the apparatus should be fairly durable if properly cared 
for. 

c. Expendable Supplies - Copies of the paper-and-pencil tests and reporting 
forms are the only expendable supplies. 

d. Replacement - Standardized tests can be ordered from their publishers. 
Broken equipment is returned to Key Education, Inc. for repair. When equip- 
ment malfunction is reported, the developer will ship replacement equipment 
immediately. All forms and tests copyrighted by the developer may not be 
reproduced withouv consent. 

e. Computer Requirements - Because KEVAS results are scored in a central 
location, there are no local computer requirements. 
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Vocational Evaluation Process 

a. Preliminary Screening - No preliminary screening is mentioned in the manual. 
Like some other assessment systems, KEVAS could be used as a preliminary 
device at the beginning of a lengthy vocational evaluation. 

b. Sequence of Work Sample Administration - Under normal circumstances, two 
examiners administer the tests. The manual recommends a cvaluator/client of 
1:4 or 1:5. In order to make the best use of time, each examiner administers 
different parts of the battery. The first test examiner gives the structured 
interview and the tests: requiring the Key Tablctop Lab. While he/she is 
testing one client, another client is being tested by the second examiner on 
the following: Hearing Screening, Vision and Color Vision Screening, Reading 
Screening, Fine Motor Skills and Social Competency. 

The order of test administration is inconsequential, as long cs all subtests are 
completed. The subtests are viewed as components that can be administered 
in any order. Because the written tests require less than 1 and 1/2 hours, 
most groups can be tested on these in one sitting. However, written test 
administration may be broken into shorter testing sessions. 

In order to stay familiar with all phases of testing, the two examiner^ a»e to 
exchange test administration duties on a routine basis. 

c. Client Involvement - Except from telling the examiners to conduct a brief 
orientation session prior to testing, thr manual contains no mention of client 
involvement during test administration. In addition, there are no instructions 
for reporting results to the client. Because KEVAS is a standardized battery 
of tests, client involvement is assumed to be minimal. 

d. Evaluation Setting - The emphasis on accurate measurement Msing psycho- 
physical devices to determine reaction time, dexterity, etc. resu^is in a lab- 
oratory like environment. 

e. Time to Complete the Entire System - Test sessions "run between 2-1/2 and 3 
hours." According to the manual, if two sessions are held during the day, 
between 16 and 20 clients could be assessed by two evaluators in one day. 

Adminstration 

a. Procedures - Test administration procedures are given in paragraph format in 
the manual, with verbal instructions separated from demonstrations by quotes; 
this makes the verbal instructions difficult to locate. Procedures for record- 
ing apparatus and perceptual test results are given during the required ex- 
aminer training. 

b. Method of Instruction Giving - There are two separate methods of instruction 
giving. The apparatus and perceptual tests are administered individually using 
a combination of verbal instructions and demonstration. The paper-a-.d-pencil 
tests are described as largely self-administered. Although not instrur 'on 
giving per se, some data are obtained from short interviews with the clie.it 

c. Separation of Learning/Performance - There is no meniic^ o7 separation of 
learning and performance in the manual per se. Given the short practice 
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sessions that usually proceed many of the tests, there could be a problem 
separating the two. 



d. Providing Assistance to the Client - On some of the apparatus tests, the 
examiner carefully monitors client performance and corrects the client as 
needed. Providing assistance to clients during the paper-and-pencil tests is 
covered during training. 

e. Repeating V/ork Samples - There are no instructions for repeating work 
samples in the manual. Since KEVAS is an assessment device, it is assumed 
that tests would not be readministered under normal conditions. 

6. Scoring and Norms 

a. Timing - In an attempt to reduce anxiety, none of the tests are timed. 
Subjects are instructed during the orientation period to "work at their own 
pace." 

b. Timing Interval - Because the tests are not timed, this is not relevant. 

c. Time Norms - There are no separate time norms per se; test results are 
recorded and sent to Key Education, Inc. for computer scoring. 

d. Error Scoring - Although errors arc recorded for some of the psychophysical 
tests, most tests are only scored by the number of correct responses. 

e. Scoring Aids - No scoring aids are used. 

f. Quality Norms - This is not relevant for KEVAS. 

g. Emphasis in Scoring - Correct responses and reaction times are emphasized. 

7. Observation of Clients 

a Work Performance - No factors are defined or observed. 

b. Work Behaviors - The manual contains some information on specific behaviors 
thai are to be observed during testing. 

c. Recording System - On one of the data recording sheets the test examiner 
can rate both client behavior and the testing environment as being "favorable" 
or "unfavorable." 

d. Frequency of Observation - This was not mentioned in the manual. 

8. Reporting 

a. Forms - Each examiner has a separate form for recording test results; check- 
lists are provided for users, when requested, to ensure that the administration 
is cotnplete. 

The projective drawings and sentence completion are reviewed, along with 
interpreted test data, by a clinical psychologist and results are summarized in 
a narrative. A bank of narrative paragraphs is used to generate personality 
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results and these are integrated into appropriate intervention recommendations 
(e.g., referral to DVR for specified services, further counseling suggested). 



b. Final Report Format - There are several report formats. The typical report 
begins with a list of the tests taken and a description of the rating scale 
used to report results: suoerior (significantly above the norm), above average 
(above the norm), average (this is divided into three separate classes: high 
average, within normal limits and low average), below average (below the 
norm) and impaired (significantly below the norm). Demographic variables 
such as education level and job history are provided. The next section lists 
the results from the interest inventory, self-directed search and the literacy 
test. Short narrative statements provide information on "intervention recom- 
mendations." Personality measures are also listed graphically. Specific voca- 
tional recommendations are listed by DOT title and code. All report formats 
are easy to read and understand. 

9. Utilitv 

a. Vocational Exploration - Because of the abstract nature of the tests and the 
formal administration procedures, KEVAS would provide very few client career 
exploration experiences. 

b. Vocational Recommendations - The training recommendations contain jobs 
and specific locally available training programs. 

c. Counselor Utilization - The counselor's report includes impairments noted and 
intervention recommendations. The impairments and intervention recommenda- 
tions provide a good basis for forming a variety of client recommendations. 

10. Training in the System 

a. Training Required - Examiners must be certified before they use the system; 
this certification is good for one year. Training costs are $300.00 per 
examiner; all travel expenses are the responsibility of the contracting agency. 

b. Training Available - Training is available from the developer. 

c. Duration - Two days of training are provided. The first six hour session is 
primarily instructional, while the second session includes demonstrated and 
supervised test administration. 

d. Follow-up - Rccertification training is conducted on an annual basis. One 
day of train, ng is conducted at a fee of $150.00. 

1 1. Technical Considerations 

a. Norm Base - There are three national data bases: normal adult population, 
normal youth population, and handicapped youth population. Samples are 
described in terms of area of country, sex, race and age. The system em- 
phasizes the development of local data bases. Key Education has published 
several descriptive statistical studies on young adults (Key Education, 1985c), 
dislocated workers (Key Education, 1985b), handicapped high school students 
(Key Education, 1986). Preliminary local norms are developed based on small 
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samplings and are refined at N = 200 or at the completion of project mile 
stones. 



b. Reliability - Reliability results for the Key Education, Inc. developed equip- 
ment are presented using a variety of methods, test-retest, split-half, KR 20, 
and Coefficient Alpha. Using sample sizes between 53 and 200, the study 
reported reliabilities coefficients of between .63 and 1.00. Most of the coef- 
ficients were in the .80's or higher. Given the shortness of most of these 
tests, these results are acceptable. 

c. Validity - There is no discussion of validation in the KEVAS manual. At 
present validation is limited to two studies predicting the job performance of 
corrections officers in Monmouth County, New Jersey (Conlon, 1985a) and the 
New Jersey Department of Corrections (Conlon, Krass and Penfield, 1985). In 
both of these studies, KEVAS results were validated against supervisor's 
performance ratings. Results indicated that the system could successfully 
predict job performance. 

Although not strictly validity, a bank of more than 600 occupations, identified 
by DOT title and code but NOT utilizing DOT criteria, wa.<^ developed based 
on labor market surveys conducted in the midwest, southeast and northeast. 

Training programs are identified based on review of the options available to 
the user. Task analyses were conducted for many occupations and for all the 
training options included to date. 

12. Reviewer's Summary and Comments 

KEVAS is designed to assess a wide variety of populations on basic perceptual and 
other skills using a combination of apparatus and paper-and-pencil tests. In this 
respect the system is similar to the Career Evaluation System and McCarron-Dial 
(Work) Evaluation System. The major advantages of the system arc its short 
administration time and the number of clients that can be assessed in one day. 
The manual is well-written and easy to use. 

13. Address 

Key Education Inc. 

673 Broad Street 

Shrewsbury, New Jersey 07701 

14. Cost 

There are two major pricing options: 

Option I - Provision of complete vocational assessment services including Certified 
Examiners to administer tests on sites provided by the contracting agency. 
Includes complete data scoring and interpretative reporting services and group 
data maintenance. Travel expenses are additional. The cost per subject is: 

Groups of 5 to 25 $100.00 per subject 

Groups of 26 to 49 $85.00 per subject 

Groups of 50 or more $75.00 per subject 
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Option II - Provision of training services for agency staff members, who then 
administer the test program. Provides training materials and conducts sessions 
cither at Key Education, Inc. or at agency site. Costs for this option are: 

Training services and materials $300.00 per examiner 

Lease of test equipment $250.00 per month 

All consumable test materials, complete scoring, data maintenance and individual 
reporting services (Graphic Report Format): 

From 1 to 99 $40.00 per subject 

From 100 to 250. $35.00 per subject 

From 251 to 499. $30.00 per subject 

Over 500 $25.00 per subject 

Group Data Analysis Services - Once a group of clients has been included in the 
database, statistical analysis can be conducted. Summary statistics and recommen- 
dations are provided in report form on request and under separate agreement. 
Costs depend upon the size of the database to be studied. 
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McCarron-Diai (Work) Evaluation System 

(McCarron-Dial or MDS) 



1. Development 

a. Sponsor - The MDS v/as originally developed by Lawrence T. McCarron and 
Jack G. Dial. It is presently being marketed by McCarron-Dial Systems, Inc. 

b. Target Group - The system can be used wit^ special education and rehabilita- 
tion populations at any level of intellectual functioning and with disabilities 
in one or more of the following aress: physical, mental, emotional or func- 
tional behavior. Target disability groups include: learning disabled, emotion- 
ally disturbed, mentally retarded, cerebral palsied, closed head injured and 
socially handicapped or culturally disadvantaged. It can also be adapted for 
use with blind and deaf persons. 

c. Basis of the System - The MDS is based on a neuropsychological theoretical 
framework. The system includes an assessment of five factors: verbal-spa- 
tial-cognitive, sensory, motor, emotional and integration-coping. These five 
factors were derived from an assessment of three dimensions: verbal and 
synthetic-spatial skills; sensorimotor skills; and emotional-coping skills. 

2. Organization 

a. Name and Number of Work Samples^ - The basic MDS consists of six separate 
instruments grouped into five factors: The six instruments included in the 
basic MDS are: 

(1) Peabody Picture Vocabulary Test-R (PPVT-R) 

(2) Bender Visual Motor Gestalt Test (BVMGT) 

(3) Behavior Rating Scale (BRS) 

(4) Observational Emotional Inventory (OEI) 

(5) Haptic Visual Discrimination Test (HVDT) 

(6) McCarron Assessment of Neuromuscular Development (MAND) - Contains 
ten short tasks to assess fine and gross motor abilities 

Scores, if available, from the following categories of tests can also be fac- 
tored into the analysis, prediction and programming model provided by the 
McCarron-Dial System: 

(1) Wechsler Intelligence Scales (WAIS, WAIS-R, WISC, WISC-R) 

(2) Academic achievement scores (WRAT, FIAT, etc.) 

(3) Supplemental instruments available from MDS, Inc. including: Perceptual 
Memor} Task (PMT), Street Survival Skills Questionnaire (SSSQ) and 
Haptic Memory Matching Test (HMMT) 

(4) New MDS paper and pencil tests which may be added or substituted for 
existing tests including: OEI-R, Emotional Behavioral Checklist (EBC) 
and Survey of Functional Adaptive Behaviors (SFAB) 



Note: The McCarron-Dial is not a "work sample system," per se. 
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b. Grouping of Work Samples - The tests, tasks and scales are grouped according 
to five factors: 

(1) Verbal-Spatial-Cognitive: PPVT (also, if available, WAIS, academic 
achievement, PMT) 

(2) Sensory: BVMGT, HVDT (HMMT may be used in conjunction with HVDT 
with visually impaired persons) 

(3) Motor: MAND 

(4) Emotional: OEI (OEI-R or EBC may be substituted) 

(5) Integration-Coping: BRS (also, if available, SSSQ; SFAB may be sub- 
stituted for the BRS) 

c. Manuals - The basic MDS comes with a system manual: McCarron-Dial Evalu- 
ation System: A Systematic Approach to Vocational Educational and Neuro- 
psychological Assessment . This manual contains an overview of the system, 
administration and scoring instructions for all of the instruments plus data cn 
combining and interpreting the results. Manuals for the BVMGT and PPVT 
are contained in the Auxiliary Kit. The HVDT, MAND, SSSQ and PMT each 
include individual manuals. The SSSQ also includes Curriculum Guides which 
give remediation strategies for deficit areas. All of the manuals are very 
detailed and thorough. In addition to adult norms, children's norms are 
contained in each manual. 

Physical Aspects 

a. Packaging of the instruments - The basic MDS is packaged in three separate 
kits (Auxiliary Kit, HVDT and MAND), each about the size of a large brief- 
case. The Auxiliary Kit contains the Bender, PPVT, OEI, BRS, reporting 
forms and the system manual. Supplementary instruments (PMT, SSSQ and 
HMMT) are also contained in separate kits. 

b. Durability - Because the evaluator or psychologist sets up the equipment and 
is present at ail times, there should be little problem with durability. 

c. Expandable Supplies - The only expendable items are the various test answer 
sheets, behavioral observation forms and report forms. No consumable mate- 
rials are used. 

d. Replacement - All replacement parts must be ordered from the developer. 
This is absolutely necessary to maintain standardization of the testing materi- 
als. 

e. Computer Requirements - The MDS has the following optional scoring and 
report writing software: Computer Assessment Program for preparation of a 
comprehensive report. Occupational Exploration Systems for relating factor 
scores to over 2,000 DOT jobs, SSSQ Computer Report for scoring, plotting 
and referencing the SSSQ curriculum guide. Individualized Trait Analysis for 
Program Planning (ITAPP) analysis of 60 specific traits derived from the MDS 
and Remedial Motor Training for providing training ac:lvities to remediate 
identified neuromotor deficits. 

These rei^Jrting programs are available for the following computers: IBM PC, 
XT and AT; Apple II, II plus, lie and lie; and TRS-80. 




Work Evaluation Process 



a. Preliminary Screening - An interview with the client and the referral source 
is urged to obtain background data on the client. 

b. Sequence of Instrument Administration - Administration begins with factor 
one and continues through factor five. The order of administration can be 
altered depending on the needs of the client. 

c. Client Involvement - Client involvement is encouraged during the assessment 
period. Upon completion, the manual recommends individual counseling to 
provide help for the client toward realistic work-training goals and expecta- 
tions. 

d. Evaluation Setting - A formal testing setting is used for each of the assess- 
ment instruments with the exception of the BRS and OEI. Behavioral obser- 
vations made on the OEI (or OEI-R) may take place in a prevocational class- 
room or sheltered work setting. Supplemental instruments such as the SFAB 
and EEC may be used iri the regular classroom or office setting. 

e. Time to Complete the Entire System - The formal testing required for the 
abbreviated battery can be completed in three hours or leis. The comprehen- 
sive battery requires the formal testing plus systematic observation for up to 
five days in a work or classroom setting. 

Administration 

a. Procedures - Instructions, materials needed, layout and scoring procedures are 
all specified in detail. Standardized tests are administered according to 
instructions in their test manuals. 

b. Method of Instruction Giving - All instructions are given orally, through 
demonstration or by total communication systems as needed. 

c. Separation of Learning/Performance - Most of the factors in the MDS are 
based on formal testing concepts which do not separate learning from perfor- 
mance. The HVDT and MAND allow for repeating of instructions, demonstra- 
tions and presentation of example items, but have no formal criterion as to 
when the instructions have been learned. Because the McCarron-Dial is not a 
work sample system per se, this aspect is not applicable. 

d. Providing Assistance to Client - The evaluator is to make certain that the 
client fully understands the instructions for each task; no assistance beyond 
that specified by the test manuals can be given during formal testing. 

e. Repeating Work Samples - All factors may be repeated as necessary if the 
evaluator questions the accuracy of the results. However, re-administration 
of many of the assessments depends upon the instructions in their individual 
manuals. 

Scoring and Norms 

a. Timing - The evaluator times the client on many tasks; some parts of the 
McCarron-Dial are not timed. 




b. Timing Interval - The tasks that are timed generally involve counting the 
number of responses or accurate observation for a specified number of sec- 
onds. 

c. Time Norms - Separate time norms arc given for four of the MAND subtests. 
The remainder of the tasks involve the combination of time and quality scores 
to form a single raw score or a performance score. 

d. Error Scoring - The quality of performance is compared to a well defined set 
of standards. 

e. Scoring Aids - No scoring aids are used. 

f. Quality Norms - See "c. Time Norms" above. 

g. Emphasis in Scoring - The system emphasises the quality and quantity of 
performance. 

Observation of Clients 

a. Work Performance - Work performance factors are identified in various rating 
scales (BRS, SFAB), in the SSSQ, and in the interpretive guidelines provided 
in the system manual. 

b. Work Behaviors - Work behaviors, as well as personal-social adjustment 
behaviors, are clearly specified and many specific work behaviors are listed. 
Most behaviors are defined in observable behavioral terms. 

c. Recording System - Three-point and five-point Likert-type scales are used to 
rate behaviors or performance factors. 

d. Frequency of Observation - The OEI (or OEI-R) requires a standard two hour 
observation period conducted each of five days. The SFAB and EBC may be 
rated from observations during the formal testing period. 

Reporting 

a. Forms - Standardized forms are included for each of the assessment instru- 
ments. Summary forms including an Individual Evaluation Profile (lEP) and an 
Individual Program Plan (IPP) are also included. The process for scoring and 
interpretation is clearly detailed in the system manual. 

b. Final Report Format - The standard format for comprehensive reporting 
includes specific scores (raw scores, MDS T-scores and standard scores); 
vocational and residential placement scores; behavioral observations; case 
history information; lists of strengths and deficits; programming priorities; and 
programming recommendations. The forms which may be used for report 
development are the Individual Evaluation Profile (lEP) and the Individual 
Program Plan (IPP). The lEP allows the user to record and profile the total 
score for each test administered. A software program. Computer Assessment 
Program (CAP), will do this automatically. The IPP allows for entry of 
subtest scores and right/left measures. These scores are then profiled for 
each of the factors; space is available to include results from other tests and 
work samples; strengths and weaknesses; program goals and objectives; aad 
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descriptive or narrative information. This reporting format has also been 
computerized by a software program. Individualized Trait Analysis for Program 
Planning (ITAPP). Both the lEP and IPP (as well as the corresponding com- 
puter programs, CAP and ITAPP) include profile graphs which visually sum- 
marize all scores and compare them to the general population mean, special 
population mean, the individual's mean, and the vocational program standard 
or T-score range. 

Supplementary computer reports and profiles are available from the SSSQ 
Computer Report and the Remedial Motor Training Program (RMT). These 
reports also contain information regarding suggested curriculum and/or strate- 
gies of intervention for specific deficits. 

9. Utility 

a. Vocational Exploration - The formalized assessment procedures required for 
the first three factors offer almost no opportunity for client vocational 
exploration. The observation period either in a sheltered workshop or on a 
job site could provide chances for exploration, but this depends on the pro- 
gram of each facility. However, the McCarron-Dial Occupational Exploration 
System (OES) generates a computerized list of DOT-coded occupations based 
on an analysis of the evaluee*s factor scores obtained from the MDS and 
WRAT standard scores. The selected occupations are listed in the OES report 
by Worker Trait Group/Occupational Aptitude Profile number, DOT number 
and job title. Vocational exploration based on the OES job listing is en- 
couraged by the system. The OES can be used as an extension of the CAP 
or ITAPP or may used as a stand-alone program. 

b. Vocational Recommendations - Emphasis is on educational and vocational 
programming, development and placement. Based on significant research 
findings, the system predicts the level of vocational and residential function- 
ing the individual may achieve after training. This enables the client and 
counselor to determine realistic goals and/or appropriate placement. The 
predicted vocational levels include 4 levels of community employment (transi- 
tional, semi-skilled, skilled and technical/professional) and 4 levels of shel- 
tered employment (daycare, work activities, low extended and high extended). 
The residential levels include community living, halfway house, group home, 
intermediate care and institutional. Examples of final reports for work, 
educational and clinical uses are provided in the various manuals. 

c. Counselor Utilization - The predicted vocational level provided by the MDS 
can be used by the counselor to establish realistic vocational goals and/or 
appropriate vocational program placement. The detailed profiling of perfor- 
mance across five areas of behavior provides the counselor with a holistic 
view of the individual's strengths and needs when compared to others in 
his/her predicted vocational range as well as general population. Specific 
needs which must be remediated and/or accommodated for successful goal 
achievement are easily identified. The counselor(s) can then address these 
specific needs in an individualized program plan. 



10. Training in the System 

a. Training Required - A commitment to pursue training is a purchase prereq- 
uisite. 
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b. Training Available - Basic training is available in work evaluation and neuro- 
psychological uses of the MDS. Workshops arc held where need indicates as 
well as in Dallas, Texas at the MDS administrative office. Advanced training 
is also available on a periodic basis. 

c. Duration - Each Basic workshop session takes three days. 

d. Follow-up - Advanced training, although not required, is recommended by the 
developer for maximum use of the system. 

1 1. Technical Considerations 

a. Norm Base - Norms for the WAIS, Stanford-Binet, PPVT, MAND and HVDT 
involve 2,000 or more observations each. Norms for the PMT, OEI, OEI-R, 
BRS, SFAB, PMT and SSSQ have been obtained on more than 500 normal and 
disabled adults each. The original normative sample for the entire system as 
well as the HMMT in work evaluation was 200. Additional samples have 
extended this number considerably. Profile revisions occur periodically to 
reflect additional norms. Adult norms on the deaf, blind and aged populations 
are now available on the HVDT, HMMT and MAND. Pertinent empirical and 
statistical characteristics of the various norm groups are given in the manuals 
and in research publications. 

b. Reliability - The results of several reliability studies are presented in the 
various manuals. Experimental methods are for the most part clearly de- 
scribed. Most data are presented in terms of ^'^^^t-retest reliability coeffi- 
cients and standard error of measurements. All reliability estimates, except 
the PPVT, are in the high .80's and .90's. 

c. Validity - A variety of validity data are presented for separate parts of the 
system as well as for the entire system. The data presented covers mostly 
construct and predictive validity. The MDS has been subjected to several 
studies (sec references) which have demonstrated its usefulness as a diagnos- 
tic instrument. 

1 2. Reviewer's Summarv_andjC!omments 

The McCarron-Dlal system was initially designed for determining the overall 
functioning level of mentally retarded and/or mentally disabled persons. Since 
that time the system's usefulness has been extended to many other populations. 
The McCarrcn-Dial should not be considered as a work sample system; rather, it 
should be thought of as a series of tests and observations that appear to be 
closely related to clinical psychology. The system uses a combination of v/idely 
accepted individually administered psychological tests and other assessments, a 
determination of fine and gross motor ability and a period of behavior observa- 
tion. In this way, the MDS depends on several different assessment methodolo- 
gies. The major feature and the main advantage of the McCarron-Dial is the 
combining of several different types of assessment tools into a single report that 
provides a comprehens'-e picture of the client. In addition to merely reporting 
results, the MDS provides diagnostic information on some of the factors. 

The system conta» s excellent norms and adequate reliability data. However, the 
real achievement is the continuing validation research that cannot be equaled by 
other evaluation or assessment systems. 
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13. 



Address 



McCarron-Dial Systems 
P.O. Box 45628 
Dallas, TX 75245 
214/247-5945 

14. Cost 



Basic McCarron-Dial System $1450.00 

Auxiliary Component 

HVDT 

MAND 

Supplemental Instruments 

SSSQ $185.00 

PMT $245.00 

HMMT $625.00 

MAND Adaptations for the Visually Impaired $ 37.75 

Software Programs 

Computerized Assessment Program (CAP) $ 775.00 

Individualized Trait Analysis for Program Planning (ITAPP) $1250.00 

Occupational Exploration System (OES) $ 775.00 

Remedial Motor Training (RMT) $ 250.00 

SSSQ Computer Report $ 225.00 



Manuals are included in the kits but may also be purchased separately. 
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Microcomputer Evaluation and Screening Assessment 

(MESA) 



Development 

a. Sponsor - The MESA was designed, developed and marketed by the Valpar 
Corporation of Tucson, Arizona. 

b. Target Group - MESA is a screening device to be used with the middle 80% 
of the population. If a person is either in the top 10% or the lower 10% of 
the population, they should be referred elsewhere for other types of assess- 
ment and/or vocational evaluation. Thus, the system has application to many 
groups: high school students, a general vocational rehabilitation population, 
vocational-technical students, prisoners, and manpower trainees. Finally, 
there are special instructions for persons with hearing losses. 

c. Basis of the System - The system is intended to produce a Worker Trait 
Profile (called the Worker Qualifications Profile in the third edition of the 
DOT) for the client or student. Therefore, MESA concepts are derived from 
the U.S. Department of Labor worker trait definitions and job analysis con- 
cepts. Individual scores on the tests and other methods of data collection 
are weighted and combined to produce a Worker Trait Profile for the client 
that is compared to performance requirements for desired training or employ- 
ment. 

Organization 

a. Name and Number of Work Samples - The MESA consists of nine major test 



(1) Hardware Screening - Includes Gross Skills, Problem Solving and Fine 
Skills, each of which contain several measures. 

(2) Physical Capacities and Mobility Evaluation - Contains: Dynamic 
Strength and Mobility Evaluation. 

(3) Vocational Interest Screening - Contains: slide-tape presentation and 
paper-pencil exercise. 

(4) Vocational Awareness Screening - Assesses client's knowledge of world 
of work. 

(5) Independent Perceptual Screening Assessment - Measures: tactual feel- 
ing, form perception, and spatial ap itude. 

(6) Talking/Persuasive Screening - Short observation of speaking and 
listening skills. 

(7) Working Conditions - Environmental preferences. 

(8) Specific Vocational Preparation - Amount of time to be spent in train- 
ing. 

(9) Computer Screening Exercises - Includes 13 subtests: Vision Screening- 
Placing and Tracking; Color Discrimination; Reading; Size and Shape 
Discrimination; Short-Term Visual Memory; Spelling, Vocabulary and 
Mathematics; Language Development; Problem Solving; Hand-Eye-Foot 
Coordination; and Reasoning Ability. (The Reading, Spelling, Vocabulary 
and Mathematics tests may be eliminated at the discretion of the 
evaluator.) 



categories: 
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The system collects 84 separate pieces of information (e.g., Placing, Lifting, 
Tool Usage and Thinking Skills, and Walking Heel-Toe) for each client. 



b. Grouping of Work Samples - The tests are grouped into the nine categories 
listed above. An alternate classification method is to divide the tests into 
computer administered, hardware/hands-on and paper-and-pencil tests. 

c. Manual - The MESA manual includes the following headings: Introduction, 
Criteria Based Assessment, Computer Set-up, Computer Exercises, Other 
Computer Programs, MESA Subtests, Administration Systems, MESA/DOT, 
Access Profile Program, Evaluee Instructions Hardware, Evaluee Instructions 
Software, Independent Perceptual Screening, Talking/Persuasive, Physical 
Capacities/Mobility Evaluation, Vocational Interest, Vocational Awareness, 
SVP/Working Conditions, Distinguishing Characteristics, Interpreting Results, 
Technical Information, Bibliography, Appendix, and Mock Reports. There is a 
separate manual for physical capacities and mobility. The manual covers all 
test administration, computer set-up and data interpretation. It is very easy 
to read and understand. The detailed, step-by-step instructions for computer 
set-up and for administering the hardware/hands-on tests are especially well- 
written. 

3. Physical Aspects 

a. Packaging of the Work Samples - Each paper-and-pencil test is independently 
packaged. The hardwarc/hands-on equipment *s used for several tests. 
Finally, the computer administered tests are all contained on floppy disks; not 
all cf the.-e have to be administered. 

b. Durability - Like most Valpar products, the apparatus is well-designed and 
appears to be durable. The computer eqcipment is less durable. 

c. Expendable Supplies - The only expendable supplies are paper-and-pencil 
tests, data recording forms, and paper for the printer. 

d. Replacement - To preserve standardization, all replacement parts must be 
ordered from the developer. 

e. Computer Requirements - The system *vill run on Apple II, He, and 11+ and 
IBM-PC and IBM-XT with 128 K RAM. The basic computer requirements are: 
one disk drive, monitor and prinfer. The MESA adds a control box, hand- 
control paddle, foot control, eye-hand-foot paddle, MESA control cards and a 
clock card. Detailed installation instructions are contained in the manual. 

4. Work Evaluation Process 

a. Preliminary Screening - There is no preliminary screening for the MESA. 

b. Sequence of Work Saa>ple Adminstration - The MESA is designed to be ad- 
minist^^red to from one to four clients at a time. It is recommended to give 
the Physical Capacities and Mobility Exercise first; then the Hardware Exer- 
cise is administered to four clients. The next step is to assign one client to 
the computer exercises, one to the Independent Perceptual Screening Exercise 
and two to the Vocational Interest and Vocational Interest Factor Exercise. 
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The clients are rotated upon completion of these tasks. The final exercise is 
the Talking/Persuasive Screening Exercise, which is group administered. 



c. Client Involvement - Clients are never really informed of the progress. The 
only involvement with the evaluator is during the instructions of each work 
sample. During the Talking/Persuasive Screening Exercise, the evaluator talks 
to the clients in a group setting. As with most other assessment or screen- 
ing devices, the goal is to collect a large amount of objective data in a short 
period of time. 

d. Evaluation Setting - The evaluation setting is intended to create an informal 
testing situation. 

e. Time to Complete the Entire System - MESA can be given to between one 
and four pccple in foui hours. 

Administration 

a. Procedures - All procedures are defined and clearly stated. The manual also 
states the materials needed, purpose, set-up, administration, scoring and data 
interpretation. 

I. Method of Instruction Giving - The initial instructions for each test are 
verbal. The Hardvare Section also used posters as visual aids. Evaluator 
demonstrations are also given. The computer administered tests are largely 
self-administered. Although reading is required for some of the paper-and- 
pencil tests, it is estimated that most items are written at the fourth grade 
level. 

c. Separation of Learning/Performance - Each work sample has a practice exer- 
cise before the timed and/or scored section begin^.. Although there are no 
specific criteria to be met, the evaluator is to "walk" the client through the 
steps to make sure the client understands. Each computer administered test 
has an extended practice section; "however, the test appears on the screen 
without an introduction." There is little separation of learning from perfor- 
mancc. 

d. Providing Assistance to C?lent - The evaluator is to assist the client with 
additional instructions and/or demonstrations until the client understands the 
procedures. Allowances are made for at least two practice sessions. 

e. Repeating Work Samples - the manual states that at the completion of test- 
ing, the MESA Summary Report should determine if retesting or additional 
testing is appropriate. It does not give specific reasons for retesting. 

Scoring and Norms 

a. Timing - The computer is self-timing and records the client's times. Com- 
puterized tests have time limits for both individual items and the total test. 
Within each test, the client begins with easy items and progresses to more 
difficult items; the test is concluded when more than a specified number of 
items are lOissed. For example, the computer administered academic tests are 
discontinued after three incorrect responses are made. The hardware/hands- 
on section is timed by the evaluator. The rest of the tests are not timed. 
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All test results are entered into the computer for scoring and report prepara- 
tion. 



b. Timing Interval - There is no separate timing interval for the computer 
administered tests. All hardware/hands-on tests, with the exception of In- 
dividual Response Screening, are timed from the end of the practice session 
to completion. The GOE and Vocational Awareness are timed based on the 
slide presentation. Finally, the academic tests can be given without time 
limits, if desired. 

c. Time Norms - No time norms are used. 

d. Error Scoring - Errors are not scored; on most exercises only correct re«?pon- 
ses are counted. Separate data recording forms are used for each section to 
score the number of correct responses, tasks co npleted, etc. are entered into 
the computer by the evaluator. All scoring criteria are clearly defined. 

e. Scoring Aids - None are used. 

f. Quality Norms - This is not relevant. 

g. Emphasis in Scoring - The emphasis is on the number of correct responses. 
In the computer administered tests the number and the pattern of correct 
responses and errors are also recorded. "The Spelling, Vocabulary and Math- 
ematics exercises are scored in a different manner than the othci computer 
exercises in MESA. In these exercises, points are awarded for choosing the 
correct response on the first attempt, with fewer points being awarded to 
correct second choices." 

Observation of Clients 



c. Recording System - The system does not emphasize the systematic recording 
of behavior observations as defined within the context of this publication. 
However, the Talking/Persuasive exercise, some of the hardware tests, anc^ the 
physical capacities test are scored from the observations of the evaluator. 

d. Frequency of Observation - The manua^ states that the evaluator can observe 
behaviors all through test administration, but it does give any information on 
how often to record them. 



a. Forms - MES/\ uses a variety of forms for recording test results and evalua- 
tor*s ratings on many of the apparatus tests. All forms are well-designed 
and appear easy to use. The computer exercises are recorded directly into 
the computer. All these dota are entered into the computer and stored on 
the clients* disk. 

A client's results can be compared to act al jobs, training programs and 
specific classes by use of the Access Profile. The employer, teacher etc. 




Specific work performance and 
work behaviors are not used in 
the MESA. 



Reporting 
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rates each trait in the Worker Trait Profile. These profiles are electronically 
matched against the client's profile. 

b. Format The MESA user has the option of selecting four computer generated 
reports: Evaluation Summary Report, Worker Qualifications Profile, Analysis, 
Screening Access Profile Report and Comprehensive Access Profile Report. 
The Evaluation Summary Report contains the following headings: Employ- 
ment/Academic Background, Academic Skills, Peroeptual/Neurological (with 
subheadings for Visual Screening, Color Discrimination, Size ancJ Shape Dis- 
crimination, Eye-Hand-Foot Coordination, Eye-Hand Coordination, Physical 
Capacities, Strength, Mobility, Fine Finger Dexterity, and Manual Dexterity) 
General Abilities (includes: Problem Solving, Visual Memory, Reasoning, Talk- 
ing/Persuasive Exercise, Independent Perceptual Screening and Instruction 
Following), Vocational Awareness, Vocational Interests (GOE and Vocational 
Interest Factors). The report is computer generated and gives the client 
scores and allows room for comments and a summary. 

9. Utility 

a. Vocational Exploration - Because of the abstract nature of the tests and 
Vork samples,** MESA offers almost no vocational information directly to the 
client. 

b. Vocational Recommendations - The Evaluation Summary Report contains 
specific testing results. The evaluator uses these results to prepare specific 
vocational recommendations. 

c. Counselor Utilization - The most powerful use of MESA results is to compare 
the client's Worker Trait Profile directly to jobs and/or training programs 
either using data obtain^^d from the Access Profile or from entering the 
client's profile into a job matching program, such as ValSEARCH (Botterbusch, 
1986). 

10. Trainin:g_in the Svstom 

a. Training Required - Training is not required but is highly recommended. 

b. Training Available - Training and certification are available through Valpar 
regional offices on a regular basis. Special training can be set up ?t a 
facility or school if desired. Training and certification are available at the 
annual Valpar National Training Institute. 

c. Duration - Both MESA training and certification are two-d?v sessions at 
$250.00 per day per trainer, plus expenses. Most training sessions are de- 
signed to accommodate 10 to i 5 participants at $50.00 to $75.00 per day, 
depending on trainer expenses. 

d. Follow-up - Advanced MESA training sessions are available at the same fee 
structure given above. The Valpar home office, regional offices and other 
certified valpar trainers are available for either telephone (800 number) or 
letter support. 
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11. Technical Considerations 



a. Norm Base - MESA was normed on 491 male and female students and em- 
ployed workers in Arizona. The students ranged in age from 10 to 33, i.e., 
fourth grade to junior college. Workers ranged in age from 20 to 55; they 
were employed in semi-skilled, skilled, clerical and professional jobs. The 
manual contains no information on the sample selection, methods used and 
some of the sample characteristics are not given. Method-Times-Measurement 
norms were also used. 

The MESA Statistical Information Report contains testing results for a group 
of learning disabled persons, honor society students and, more important, on a 
group of 3881 subjects. Most of the sample characteristics are provided. 

b. Reliability - The manual contains one test-retest reliability study for 22 of 
th- tests over a one month period. The reliabilities ranged from .79 to .96. 
Although these appear adequate, there is no information as to sample size, 
testing conditions, etc. Thus, any interpretation of these results must be 
tentative. 

c. Validity - MESA developers took several approaches to "validate" the system. 
Some of these developments centered on a criterion-referenced validity ap- 
proach. MESA tests and exercises were related by expert judges to the 
ratings on the scales that make up the Worker Trait Profile. The manual 
reports high agreement on these ratings. The Report also contains data on 
"construct" and "criterion" validity. A series of 17 tables "present the cor- 
relation of the MESA subtests to each of the 1 7 factor scores in the Worker 
Qualifications Profile." No explanation is given on the development of these 
data, nor are any suggestions offered on interpretation. The bulk of the 
criterion related validity section presents "data on how MESA scores on 
specific factors of the Worke: Qualifications Profile compared to the scores 
on those same factors in the JOB BANK 12,375." Although interesting, it is 
hard to interpret these results as criterion related validity. 

12. Reviewer's Summary and Comments 

The MESA is designed to provide a vocational assessment of four persons in four 
hours and to provide accurate assessment for the middle 80% of the population. 
To the developers* credit the system does not claim to be all things to all peonle. 
The manual relates MESA limitations to the need for a more comprehensive voca- 
tional evaluation. The system has a definite place in schools, manpower programs 
and other programs where a quick, yet comprehensive assessment is needed. 
Because the results are given using the Worker Qualification Profile (i.e.. Worker 
Trait Profile), they can be easily used for input into most computerized job 
matching systems. Another advantage of this profile is that evaluators familiar 
with DOL concepts will be ab!'? to interpret MESA results with additional training. 

As with other Valpar products, the hardware is designed for ease of administra- 
tion ?nd scoring; given proper care, it should last a long time. The computer 
administered tests, however, do present some problems. This reviewer has heard 
Gome complaints about computer set-up during the last few years. Most of these 
problems have been corrected. 
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The major concerns with the system are technical. The manual presents a single 
reliability study almost without any explanation. While these results appear 
adequate, additional reliability studies need to be performed. 

Validation presents a serious problem. MESA^'s major validation is to compare 
results with DOT variable statistics and final scores on the test-generated Worker 
Qualification Profile. The two manuals contain no studies where the M£SA was 
validated against an outside criteria such as prediction of training or job success. 
In addition, given the developers' emphasis on construct validity, no factor analy- 
sis results or correlations with other tests,, work samples, etc. for the various 
subtests are reported. 

To end this review on a positive note, the system's use of the Access Profile to 
obtain data about local jobs and/or training programs is a good step. The in- 
clusion of this process should make users realize that the most powerful use of a 
system is in its ability to predict success in real local jobs and training programs. 

13. Address 

Valpar International Corp. 
P.O. Box 5767 

Tucson, Arizona 85703-5767 

14. Cost 

The MESA is available both with and without a computer. The hardware can be 
purchased to test 1, 2, or 4 persons at a time. 

A. With computer 

Apple, 64 K RAM, 2 disk drives, color monitor, printer 

4 station 
2 station 

1 station 

IBM-PC, 128 K RAM, 2 disk drives, color monitor, printer 

4 station 

2 station 

1 station 

B. Without computer (specify Apple or IBM) 

4 station 

2 station 
1 station 

15. References 

Fry, R. (1984). MESA (Microcomputer Evaluation and Screening Instrument). 
Vocational Evaluation and Work Adjustment Bulletin. 12(2), 67-70. 
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$10,100.00 
$8,325.00 
$7,325.00 



$11,700.00 
$9,925.00 
$8,925.00 



$6,750.00 
$4,975.00 
$3,975.00 



Microcomputer Evaluation of Career Areas 

(MECA) 



L Development 

a. Sponsor - The system was developed by Mr. Terry Schmitz of the Conover 
Company; some materials were developed by the Special School District of St. 
Louis County, Missouri. 

b. Target Group - The manual and other literature clearly state that MECA was 
designed only for use in schools. The system provides occupational explora- 
tion experiences for junior and senior high schools. Within this general 
target group, MECA focuses on disadvantaged and special needs students. 

c. Basis of the System - The following quote is taken from introductory materi- 
als on MECA: "Each career kit in the MECA system was developed on the 
multiple-trait factor approach to assessment; that is, during the assessfwent 
process, several trait factors are assessed at one time. This approach is 
sometimes referred to as a job sample appioach to assessment. These trait 
factors are keys to a specific job title which interfaces with a major career 
interest area within one of the sixty-six worker trait groups (sic) of Th^ 
Dictionary of Occupational Titles. " 

2. Organization 

a. Name and Number of Work Samples - The system presently contains 15 work 
samples. Each sample consists of three tasks (called *'work samples" by the 
developer). The work samples and their component tasks are as follows: 

(1) Automotive - Repairs a wheel cylinder, replaces points and a condenser, 
and adjusts points. 

(2) Building Maintenance - Repairs a faucet, prepares electrical cable, and 
installs new electrical cable. 

(3) Cosmetology - Gives a manicure, curls long hair, and gives a facial. 

(4) Graphic Design - Uses a lettering guide, uses transfer lettering, and 
operates an air brush. 

(5) Custodial Housekeeping - Dust mops a floor, cleans windows, and cleans 
furniture. 

(6) Electronics - Checks electronic circuits, wires a circuit board, and 
solders resistors on a circuit board. 

(7) Small Engines - Services the air cleaner, services the ignition system, 
and cleans the cooling system. 

(8) Food Service - Sets a restaurant covei, takes orders, and pr^jpares 
food. 

(9) Health Care - Wraps an arm with an elastic bandage, and determines 
temperature, pulse and blood pressure. 

(10) Business and Office - Files, takes messages, and types letter. 

(11) Manufacturing - Sorts and packag-^s, assembles note pads, and assembles 
using exploded view drawings. 

(12) Distribution - Fills customers' orders, ships customers' orders, and sets 
up a display. 
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(13) Construction - Makes a butt joint, makes a miter joint, and installs a 
lock assembly. 

(14) Horticulture - Plants seeds, mixes high porosity soil, and tests soil. 

(15) Computers - Date entry, travel agent, and computer operator. 

In addition, each of the 15 work samples has two supplementary exercises. 
The Learning Activity Packet contains mathematics, reading and other exer- 
cises specifically related to the contents of the work sample. For example, 
the Small Engines contains the following exercises: Name the Part, Small 
Engine Mathematics, Read It Righi, Small Engine Ignition System, Electro- 
math, Small Engines Test, Small Engine Cooling System, and More Small 
Engine Math. The Math on the Job^ contains exercises relating arithmetic to 
job experiences. 

b. Grouping of Work Samples - Each work sample is independent. 

c. Manual - T,*ere is a separate manual for each work sample; each uses the 
same format and each contains identical sections on general topics such as 
job classifications, use of the microcomputer, and administration. Each man- 
ual contains the following sections: Introduction, Work Sample Descriptions, 
Academic Reinforcement Descriptions, Job Classifications, Administration - Use 
of the Microcomputer, Administration - General Information, Administration- 
Work Samples, Administration - Academic Reinforcement, Administration- 
Management System, Interpretation of Results, Definition of Terms, Repro- 
ducible Forms, Order Form, and Appendix. Each manual is very complete and 
contains full system details. For example, the narration for each "frame" of 
the computer-administered instructions is given. All tools and materials are 
clearly described. 

3. Physical Aspects 

a. Packaging of the V7ork Samples - Each work sample is packaged separately in 
a locked fiber board and metal box. 

b. Durability - The computer software is the most sensitive part of the system, 
Conover has a free replacement policy for all damaged software. Most other 
equipment appears durable. 

c. Expendable Supplies - Each work sample requires a variety of expendable 
supplies. For example, the horticulture work sample requires: paper towels, 
soil test kit, peat moss, sand, vermiculite, lime, gypsum, superphosphate, 
magnesium sulfate, trace elements mix, seed starting soil, marigold seeds, soil 
mix and a grease pencil. The developer estimates the average cost of expen- 
dable items for all 15 work sampler per student to be less than $8.00. 

d. Replacement - All supplies can be ordered from Conover; an order form is 
included in each manual. Some expendable supplies are locally available. 

e. Computer Requirements - Each work sample is administered on an Apple II+, 
He or lie computer with at least 48 K RAM, one disk drive, and a monitor. 



^ The Nfeth on the Job series was developed by the National Center for Research 
in Vocational Education of the Ohio State University. 
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Although any monitor will operate, a color monitor is recommended. All 
software is contained on self-booting floppy disks. If audio administration is 
desired, a cassette tape recorder with a remote jack is needed. The link 
between the computer and tape recorder is a Cassette Control Device devel- 
oped by Hartley Courseware. 

4. Vocational Evaluation Process 

a. Preliminary Screening - This is one of the few topics not covered in the 
manuals. 

b. Sequence of Work Sample Administration - Because each work sample is 
independent, this section is not applicable for MECA. 

c. Client Involvement - The client is involved in most of the work sample 
process. Through the use of another Conover product, the Career Planning 
System (CPS), the client selects occupational areas for further exploration. 
Within each separate MECA work sample, the client rates his/her interest and 
performance on each section of the work sample. Finally, the CPS can be 
used to relate work sample results to educational planning. 

d. Evaluation Setting - Because the MECA is designed specifically for schools, a 
classroom atmosphere is assumed. 

e. Time to Complete the Entire System - This reviewer estimates that the three 
tasks in each work sample can be completed in about two hours. If the 
Learning Activity Packets are used, this time would be extended by about 
another hour. 

5. Administration 

a. Procedures - All materials are clearly specified; photographs are used to 
insure proper set-up. The Learning Activity Packets and the Math on the Job 
series are optional programs, administered at the discretion of the evaluator. 
Individual Learning Activity Packets can be administered; these act as both a 
test and a review of the work sample. 

b. Method of Instruction Giving - Although there are two different methods, 
both are dependent on the computer. Prior to the start of computer instruc- 
tions, the evaluator gives a brief introduction. In both methods all visual 
instructions are contained on graphic illustrations displayed on the computer 
monitor. In the first method, written instructions are displayed below each 
"frame"; these instructions are written ai about the fourth grade level. A 
second method is available for non-readers. Here a cassette audio tape, 
containing instructions identical to the written instructions, is synchronized 
through the use of the Cassette Control Device. There arc also numerous 
evaluator check points; these could provide a method of additional instruction. 

The Learning Activity Packet is computer administered and scored. The Math 
on the Job module is self-administered using a workbook approach. 
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c. Separation of Learning/Performance - Because the work samples do not con- 
tain r.ny practice trials or specific criteria to be reached, there is no separa- 
tion of learning and performance. 

d. Providing Assistance to the Client - There are several evaluator check points 
in each task. Although intended primarily for eiror scoring, they could also 
be used to provide needed assistance. 

e. Repeating Work Samples - This is one of the few topics not covered in the 
manual. 

6. Scoring and Norms 

a. Timing - The evaluator times the student. 

b. Timing Interval - Separate times are taken for each task. Timing usually 
begins v/h^n the student "sits down and starts performing on this work sam- 
ple.... The completion of timing is ai the end of the v/ork sample.** 

c. Time Norms - Time standards are given in ranges (e.g., *'I7-21 minutes**); 
these are converted to a five point rating scale. 

d. Error Scoring - At each evaluator checkpoint, the number of **major** and 
**minor** errors are recorded. These errors are clearly defined in the manual. 
One point is deducted for minor errors; two for major errors. 

c. Scoring Aids - No scoring aids are used. 

f. Quality Norms - As with time norms, the number of errors on each task are 
converted to a five point rating scale. 

g. Emphasis in Scoring - Time and quality are given equal weight. 

7. Observation of Clients 

a. Work Performance - MECA identifies and defines 12 *'Primary Skill Factors,** 
such as, size discrimination, use of hand tools, and motor coordination. 
Various combinations of these are to be observed in each work sample. 

b. Work Behaviors - Two separate classes of work behaviors are identified. 
First, 13 **Secondary Overall Skill Factors** (e.g., ability to follow written 
instructions, following a model, and care in handling) are observed. As with 
work performance, different combinations of factors are observed in each 
work sample. Second, the original MDC Behavior Identification Format 
(Esser, 1974), which contains 22 separate work behaviors, is used for overall 
observation.^ 
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* This original version of the MDC Behavior Identification Format was developed 
under federal funding and was not copyrighted. The Conover Company, however, 
obtained the Materials Development Center's permission to use the Format. There is 
no commercial relationship between MDC and Conover. 
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c. Recording System - The Primary Skill Factors and Secondary Overall Skill 
Factors are rated on a five point scale, '''he MDC Behavior Identification 
Format is rated on a six point scale» with ihe major ratings being "accep- 
table," "selective placement," and ''change needed " 

d. Frequency of Observation - Thl:i Is not mentioned in the manual. 

8. j^gportins 

a. Forms - Two separate forms are used with each work sample. The Interest 
Reaction Fcim, completed by the student, contains ratings of interest and 
ability for cu''*' task as well as spaces for recording the results of the Learn- 
ing Activity lacket. A teachers' or evaluators' form lists the Primary S*<ill 
and Secondary Overall Skill Factors relevant for the particular work sample, 
defines the errors for each task, and contains ratings for time and quality. 
The other side of this form contains the MDC Behavior Identification Format. 

b. Final Report Format - Although no final report format is suggested by the 
MECA work samples per sc, another CoTiOver product, the Vocational Report 
Printout, can be used for a final report. The following information is en- 
tered: perianal and educational data, academic competencies, CPS data, and 
MECA results. These data are then combined and the resulting report can be 
used for future vocational planning. 

9. Utility 

a. Vocational Exploration - MECA is designed as a vocational exploration system. 
The system definitely z^jhieves this stated purpose. 

b. Vocational Recommendations - Because the MECA is intended for srhool 
systems, vocational recommendations center around planning future educational 
goals. 

c. Counselor Utilization - The report is designed for the teacher and school 
guidance counselor. 

10. Training in the System 

a. Training Required - Training is not required for purchase. 

b. Training Available - Training is available from Conover and some cf its 
dealers. 

c. Duration - CPS training takes about two hours; MECA training about four 
hours. The cost for CPS is J 100.00 and MECA is $250.00, these costs do not 
include travel expenses. 

d. Follow-up - Follo^v-up is available from Conover and its dealers. 

11. Technical Considerations 

a. Norm Base - No norms or predetermined time standards are given or even 
mentioned in the mciuals. 
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b. Reliability - No information available. 



c. Valici^;/ - No information available 

12. Revic'.yer's Summary and Comments 

The MECA is a series of 15 work samples designed for vocational exploration; the 
target population is school populations. MECA kits are very well designed, in- 
clude durable tools and equipment, and are priced much lower than most other 
occupational exploration orientated work samples. The use of computer tech- 
nology allows for instruction using well-designed graphics, reading from the 
computer screen, and if needed, audio cassette. These computer given instructions 
are the heart of the system, as well as the reason for comparatively low per unit 
costs. The system also features closely related software and print media that 
relate arithmetic and reading comprehensica to the specific work sample. 

The system's main flaw is the complete lack of norms, reliability, and validity. 
While it may be too much to expect empiric /alidity from an essentially occupa- 
tional exploration device, the developer neeuS to obtain normative data to deter- 
mine test-retest reliability, and to include information on content validity for 
each work sample. Such information should relate each task within each work 
sample to similar or identical tasks required on specific jobs. 

13. Address 

The Conover Company 

P.O Box 155 

Omro, Wisconsin 54963 

14. Cost 

There are two separate purchase options for purchasing individual work samples: 

A. Software and administration manual for each work sample $250.00 

(per work sample) 

B. Complete Work Sample - includes software (3 task disks and Learning Activity 
Packet), administration manual, related Math on the Job, hardware, hand tools 
and supplies in a package case: 



Automobile $550.00 

Building Maintenance $550.00 

Graphic Design $55C.OO 

Cosmetology $550.00 

Custodial Housekeeping $550.00 

Electronicb $595.00 

Small engines $725.00 

Food Service $550.00 

Business and Office $450.00 

Health Care. $450.00 

Manufacturing $550.00 

Construction $595.00 

Distribution $595.00 

Hordculture $495.00 
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Computers $350.00 

Individual Learning Activity Packets $69.95 

Cassette Control Device $79.95 

15. References 

None presently available. 
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1. Development 

a. Sponsor - The system was developed by the ICD - International Center for 
the Disabled. However, support was obtained from the HEW Rehabilitation 
Services Administration to collect normative data. 

b. Target Group • The system was primarily aimed at a general rehabilitation 
population, but it can also be used with special education students, the disad- 
vantaged, and adult offenders. It can also be used with learning disabled and 
higher level mentally retarded adolescents and adults. It is not intended for 
use with persons who are above average in intelligence. 

c. Basis of the System - The system is basically a group aptitude test that uses 
work sample methodology to measu^-^ seven aptitudes as defined by the fourth 
edition cf the Dictionary of Occuoi onal Titles and the General Aptitude Test 
Battery (GATB). The statistical basis are studies on the factor analysis of 
several work samples and concurrent validity studies. 

2. Organization 

a. Name and number of Work Samples - The system contains 13 work samples 
which measure eight specific aptitudes, plus General Learning Ability or G. 
The work samples are, however, organized into five major groups of what can 
be thought of as second order factors. The primary aptitude(s) and the 
DOT/GATB abbreviation for each work sample are given in the parentheses: 

(1) Motor - Electronic Connector Assembly (F-finger dexterity); Bottle 
Capping and Packing (M-manual dexterity); and Lamp Assembly (K-motor 
coordination). 

(2) Spatial - Blueprint Reading (S-spatial reasoning); and Graphics Illustra- 
tion (S-spatial reasoning; K-motor coordination). 

(3) Clerical Perception - Filing (Q-clerical perception; K-motor coordination); 
Mail Sorting (Q-cIerical perception; M-manual dexterity); Zip Coding (Q- 
clerical perception); and Record Checking (Q-clerical perception). 

(4) Numerical - Making Change (N-numerical reasoning); and Payroll Com- 
putation (N-numerical reasoning). 

(5) Verbal - Want Ads Comprehensions (V-verbal comprehension); and Mes- 
sage Taking (V-verbal comprehension). 

It must be noted that four work samples (Want Ads Comprehension, Zip 
Coding, Blueprint Reading, and Payroll Computation) have alternate forms to 
prevent cheating during administration and for possible use during retesting. 

b. Grouping of Work Samples - The work samples are grouped according to the 
fWe aptitude areas listed above 

c. Manual - The system contains several manuals. A general administration and 
scoring manual, a manual for the group discussion program, a separate manual 
for each work sample, and a technical manual. Each of 13 work sample 
manuals contains the following: description, materials, setup, administration, 
scoring, and sample forns. All manuals are well written and detailed. 
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Physical Aspects 



a. Packaging of the Work Samples - All work samples are individually packaged; 
no parts are usf,d by more than one work sample. 

b. Durability - The hardware is durable and becauSv the system uses little 
complex equipment, minimal equipment replacement can be expected. 

c. Expendable Supplies - Wire in the Lamp Assembly Work Sample and the 
various paper forms arc the only expendable supplies. 

d. Replacement - Forms can be ordered from Micro-TOWER or locally duplicated. 
The cassette administration tapes must be ordered from the distributor. 

e. Computer Requirements - Not applicable. 
Work Fvaluation Process 

a. Preliminary Screening - No preliminary screening is required prior to the 
administration of Micro-TOWER. The manual states, however, that a period 
of general orientation to the system should be given prior to work sample 
administration. 

b. Sequence of Work Sample Administration - The manual contains several sug- 
gested schedules for administration of the work samples and for group di'jcus- 
sion. Thes'*. schedules are only suggestions and the work samples do not have 
to be given in any set sequence. However, because the Want Ads Comprehen- 
sion Work Sample tests the ability to read and understand English, it is 
usually first. Within each work sample a carefully defined sequence is fol- 
lowed and all instructions to the clients zr^ recorded on a cassette tape. 
The first step is the Dresentation of a series of occupational photo^ illustrat- 
ing jobs requiring the skills assessed by the work sample. Each wcrk sample 
provides an untimed learning/practice period which includes taped instructions, 
visual illustrations, evaluator demonstrations, and an opportunity for clients to 
practice. During this period, the cassette tape automatically stops at presel- 
ected places so that the evaluator can give additional instructions, etc. The 
evaluator is also free to stop the tape at any time if additional help is need- 
ed. After iiiiS learning/practice period coniCS the evaluation period. Here 
clients work entirely on their own without any help. After completion of the 
ta.sk, the clients fill out a self-report from rating their interest and per- 
ceived ability. 

c. Client Involvement - Micro-TOWER emphasizes client involvement, which is 
accomplished in several ways. Prior to administration of the work sample, 
occupational information is provided; during the instruction period, the evalu- 
ator stops at ''sveral points to answer questions and provide additional in- 
structions. The greatest client involvement is during the group discussion 
program. Here client values, interests, needs, etc., are discussed. Suggested 
activities (e.g., job values, lifelines, choose your supervisor) are provided in a 
sepaiate manual. Clients also receive formal feedback of their performances 
on the work samples. 

d. Evaluation Setting - The evaluation setting could best be described as a 
combination of a formal testing situation and a group counseling environment. 




The Micro-TOWER is best administered in a room that is separate from the 
rest of the evaluation unit; a "U" shaped table arrangement is suggested. 
These factors add up to the formal testing atmosphere. 

e. Time to Complete the Entire System - Total testing time is about 15 hours; if 
group discussions are included, the total evaluation takes from 19 tu 20 hours. 
Depending on what schedule is used, the battery can be administered in 
between three and five days. The manual contains several suggested sched- 
ules which vary in the number of hours per day that the work samples are 
administered and in the presence and duration of the group discussion periods. 

5. Administration 

a. Procedures - Genera! administration procedures are described in the overall 
manual. The specific manual on each work sample contains detailed instruc- 
tions on materials, layout, administration, scoring criteria, etc. All proce- 
dures are given in great detail. 

b. Method of Instruction Giving - Instructions are given by several methods. 
Each work sample begins with a series of large photographs showing jobs 
requiring skills related to the work sample. The major instructional method, 
however, is a separate audiocassette tape fo'' each work sample which is 
coordinated with evaluator's demonstrations. This tape is programmed to stop 
at certain critical points so that tAe evaluator can provide help, give addi- 
tional explanations, or check the results of the pr:.ctice exercises. The 
system emphasizes standardized instructions and timing and uses the audiotape 
as the major means of insuring this standardization. No written instructional 
materials are used. However, to complete some of the verbal and clerical 
tasks, a third to fourth grade reading level is required. In summary, ther'", 
arc five steps in each work sample; (1) occupational orientation, (2) basic 
instructions, (3) practice period, (4) timed evaluation, and (5) completion of 
self-evaluation. 

c. Separation oi Learning/Performance - The system places a great deal of 
emphasis on separation of learning from performance. Each work sample 
contains a practice period during which the clients must reach certain infor- 
mal criteria. The evaluation period is timed and is only begun once the 
clients have understood and practiced the task. 

d. Providing A^.sistance to the Client - Extensive assistance is provided during 
the learning/practice period. None is given during the actual evaluation 
period. 

e. Repeating Work Samples - The manual contains no instructions or guidelines 
for repeating work samples. The only reference to rtjadministration is made 
in regard to the use of alternate forms for four work samples. 

6. Scoring and Norms 

a. Timing - The evaluation period on each wt . sample is timed using a cassette 
tape. The tape tells the clients to "begin," then runs through a number of 
minutes of blank tape and then tells the client to "stop.** This procedure 
insures accurate timing. 
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b. Timing Interval - Timing is for a specified period within each work sample. 
Clients do not continue until they have completed the task. 

c. Time Norms - No time norms are used in this system. The score for each 
work sample is the number of correct responses; report forms also provide 
space for recording the number attempted. 

d. Error Scoring - A separate form is used for each work sample to score the 
number of correct responses, pieces completed, etc. The entire product is 
scored for each work sample; there a*c no random checks. The raw scores 
for each work sample are recorded on the "Summary of Work Sample Perfor- 
mance" sheet. Quality standards are carefully defined. 

e. Scoring A?.ds - Some use is made of scoring aids. 

f. Quality Norms - The raw scores for each work sample are compared to the 
desired norm group. A scale is used to convert the scores into one of five 
possible ratings. These ratings are based on percentile norms (which are 
given in the technical manual), one rating for each 20 percentile points. 
Thus, a very high rating means that the client scored above the 81st percen- 
tile. Norms for 19 different groups as well as MODAPT industrial standards 
are available. 

g. Emphasis in Scoring - The emphasis is on the quality of work produced within 
a specified time period. 

Observation of Clients 

a. Work Performance - No specific work performance factors are defined in the 
manual or listed on the "Behavioral Observations" form. For each work 
sample there is a space for the evaluator to record general comments; the:-e 
are no suggestions in the manual as to what these should cover. 

b. Work Behaviors - Five work behaviors (i.e., understanding instructions, atten- 
tion span, work attitude/motivation, need for individualized help, and efficien- 
cy) are listed on the "Behavior Observations" form; these are not dciined in 
behavioral te^ms. This form also includes a section on "general behavior" 
containing items like appearance, physical problems and self-image. The 
evaluator is to make short notes for all of the "behaviors" listed on this 
form. 

c. Recording System - No rating system is used for any of the items on the 
"Behavioral Observations" form. However, a six-point scale is used for gen- 
eral and work behaviors on the Summary Report Form. 

d. Frequency of Observation - Observations are to be made during the training 
phase, during the performance of the work sample, and during group discus- 
sion. While no schedule for ""requency is specified, it would appear that 
frequent observations are expected. 

Reporting 

a. Forms - The Micro-TOWER uses a variety of forms. This includes a raw 
score form for eacn work sample, the "Behavioral Observations" form men- 
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tioned above, a "Summary of Work Sample Performance" form, an attendance 
form, as well as reporting forms. The client completes a "Client Interest and 
Perceived Performance" form after the completion of each work sample; there 
is also a summary sheet for this form. 

b. Final Report Format - There are three forms used for reporting. The first is 
a profile sheet based on percentiles that gives the client's results on each 
wo/k sample on a scale from "much below average" to "much above average." 
The second is a narrative summary report format which may or may not 
include the forms mentioned under 7. a. Finally, there is a "Recommenda- 
tions" form which uses a checklist format to cover ^opics such as special 
training, individual attention, and vocational recommendations. 

9. Utilitv 

a. Vocational Exploration - The information given at the beginning of each work 
sample is designed to make clients aware of what jobs are related to the 
aptitude(s) being measured by the work sample. Many of trie topics covered 
in the group discussions center on relating personal needs to job demands and 
occupational interests. 

b. Vocational Recommendations - The system elates aptitudes to Worker Trait 
Groups that require aptitude patterns simila. iO those of the client. Thus, in 
making recommendations, the evaluator would match client's aptitudes with 
those required by the Worker Trait Groups. This process would be further 
broken down according to interests, interpretations from behavior observa- 
tions, and the results of group discussions. These recommendations would be 
written in narrative form in the narrative summary report. 

c. Counselor Utilization - the Micro-TOWER has two major uses. The first is to 
present a relatively accurate assessment of job related aptitudes in a fairly 
brief period of time. The second is to be a first or screening step in an 
extended period of evaluation. ICD, for example, uses the Micro-TOWER as a 
preliminary to more extensive evaluation systems. 



10. Training in the System 

a. Training Required - Although formal training is not required, it is desirable. 

b. Training Available - On-site training programs are available. 
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c. Duration - Two days. 

d. Follow-up - Available. 
1 1. Technical Considerations 

a. Norm Ba.se - Industrial standards through MODAPTS are available. Psycho- 
metric norms are available on a total of 19 groups. Some of the major 
groups are: general agency rehabilitation clients, males, females, Spanish- 
speaking, left-handed persons, physically disabled, psychiatrically disturbed, 
brain damaged, cerebral palsied, students in special education, the d.jad- 
vantaged, recovering drug abusers, recovering alcoholics, and adult offenders. 
G.'oup sizes rai.ge from 40 to 1300. Most sample characteristics are adequate- 
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ly described. Purchasers of the system can receive help from ICD in develop- 
ing local norms. No employed worker psychometric norms are used. 



b. Reliability - The Technical Manual provides data on the reliability of the 
Micro-TOWER wor!c samples. The coefficients range from .74 to .97. The 
data was based on test-retest, alternate forms, and internal consistency 
estimates. These estimates are adequate. 

c. Validity - Although a factor analysis revea'ed a large general factor, there 
was also evidence for grouping the work samples into the five aptitude areas. 
The construct validity of the work sample battery is supported by examination 
of the 'ntercoi relations of the Micro-TOWER work samples. Correlations are 
also available with the factors from the General Aptitude Test Battery 
(GATE). All data are reported in the Technical Manual. One study providing 
positive evidence of Micro TOWER'S use in decision-making compared the 
recommendations made after four additional weeks in TOWER. There was a 
74% agreement on vocational recommendations, suggesting that decisions can 
be reached in a much shorter time for many individuals (Reinert & Loeding, 
1978). 

12. Reviewer's Summary and Comments 

Micro-TOWER may best be described as a group aptitude battery that uses work 
sampling techniques as the assessment method. The system claims to measure 
seven of the nine aptitudes that are used in the DOT. The system has the ad- 
vantage of being group administered in a fairly short period of time, thus making 
maximum use of evaluator time. The system attempts to go beyond the mere 
assessii^ent of aptitudes by providing occupational information and group discus- 
sion. Adequate norms are available, except for employed workers. The system 
generally takes a standardized, psychological test approach with emphasis on 
carefully controlled administration conditions, the separation of learning from 
performance, and the reporting of results in terms of percentiles. One major 
problem with the system is the lack of thorough behavioral observational materi- 
als. AVtiother possible problem is the converse of the advantages of a group 
admiiwotcred test - the evaluator may not be able to provide the lient with the 
one-to-one relationship that is needed for some severely disabled peroons. 

13. Address 

International Center for the Disabled 

Att: Micro-TOWER 

340 Ea.st 24th Street 

New York, x>Iew York 10010 

14. Cost 



The cost of the Micro-TOWER depends primarily upon the number of clients being 
tested in the group. Each client requires a complete set of equipment. An 
additional se^ of equipment is needed for the evaluator. Prices are available for 
group sizes from 4 to 30, for example: 
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Number of Persons Tested Per Group 



Price 



4 
7 

10 

20 
30 



$ 8,737.00 
9,925.00 
11,113.00 
15,073.00 
19,033.00 



The above prices include all equipment, forms to test 100 clients per work sample, 
one set of evaluator's equipment for each work sample, a cassette playback and a 
cue-stop system, table easels and photo books. 
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Philadelphia Jewish Employment and Vocational Service 

Work Sample System 



(JEVS) 



1. Development 

a. Sponsor - Originally developed for the Manpower Administration of the U.S. 
Department of Labor for use in WIN and CEP programs, the JEVS has been 
refined by the Philadelphia Jewish Employment and Vocational Service. 

b. Target Group - Initially designed for the disadvantaged, the system has been 
used in the last several years as an assessment device for special needs 
populations. 

c. Basis of the System - The present basis is the Work Group system of the 
fourth edition of the Dictionary of Occupational Titles and the 1979 Guide for 
Occupational Exploration (GOE). The philosophical basis is a trait-factor 
approach between common aptitudes and behavioral demands of the Work 
Groups and wor.^ samples. 

2. Organization 

a. Name and Number of Work Samples - The JEVS contain 28 different work 
samples. (The purchaser receives a total of 48 separate work samples, 20 of 
which are duplicates of the most used work samples.) The 28 work samples 
are referenced to 12 Work Groups. Most of the 28 work samples arc used in 
more than one Group: 

05.03 - Engineering Technology - Condensing Principle 

05.05 - C-aft Technology - Blouse/Vest Making, Pipe Assembly, Resistor 
Reading, Nail and Screw Sort, Lock Assembly, Telephone Assembly 

05.09 - Material Control - Computing Postage, Nail and Screw Sort, Filing by 

Numbers 

05.10 - Crafts - Resistor Reading, Telephone Assembly, Metal Square Fabrica- 

tion, Ladder Assembly, Union Assembly 
05.12 - Elemental Work Mechanical - Hardware Assembly, Grommct Assembly 

06.02 - Production Work - Telephone Assembly, Hardware Assembly, Metal 

Square Fabrication, Grommet Assembly 

06.03 - Quality Control - Nail and Screw Sort, Collating Leather Samples, Nut 

Packing, Tile Sorting 

06.041 - Elemental Work: Industrial - Belt Assembly, Grommct Assembly, 
Sign Making, Budgette Assembly 

06.0411 - Elemental Work: Industrial - Collating Leather Samples, Nut Pack- 
ing, Washer Threading, Nut, Bolt and Washer Assembly 

07.02 - Mathematical Detail - Computing Postage, Payroll Computation, Ad- 

ding Machine 

07.03 - Financial Detail - Computing Postage, Payroll Computation, Adding 

Machine 

07.05 - Records Processing - Filing by Letters, Proofreading, Filing by Num- 
bers 

07.07 - Clerical Handling - Filing by Numbers, Rubber Stamping 
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b. Grouping of Work Samples - The work samples are organized into 12 WorK 
Groups for reporting and interpretation purposes. 



c. Manual - The Work Sample Evaluator*s Handbook contains detailed administra- 
tion and scoring instructions as well as numerous photographs to illustrate 
proper setup and common errors. The manual is well written and easy to 
follow. 

Physical Aspects 

a. Packaging oi the Work Samples - Each work sample is packaged individually; 
no tools or parts are shared with other samples. 

b. Durability - The system uses common tools and materials that should be very 
durable. The one exception is the telephone. 

c. Expendable Supplies - In addition to referral, report, and other forms, the 
major expendable supplies are: fabric, paper pads, sheet metal, and string. 
While these supplies should be available locally, they can also be purchased 
from the developer. 

d. Replacement - Most tools and equipment can be locally purchased; other items 
(e.g., colored chips) are available from the developer. 

c. Computer Requirements - Not applicable. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The work samples are administered 
in order of complexity beginning with Nuts, Bolts, and Washer Assembly and 
ending with Condensing Principle Drawing. If a client is obviously not able 
to complete the work samples at any one level, more complex work samples 
are usually not administered. 

c. Client Involvement - A client orientation is given at the beginning of work 
sampling, a motivational group interview at the end of the first day and a 
structured Feedback interview at the completion. Since work sample ad- 
ministration resembles realistic work setting, interaction between client and 
evaluator occurs between work sample administration and during the above 
sessions. 

d. Evaluation Setting - A realistic work atmosphere and setting are stressed in 
the ma. ual. 

e. Time to Complete the Entire System - The average client takes six or seven 
days to complete the 28 work samples. 

Administration 

a. Procedures - The layout is clearly described and photoglyphs are used to 
insure proper s.etup. The materials listed for each work sampi? are not listed 
at the beginning of the instructions for that work sample. The evaluator 
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provided with a list of materials for each work sample as detailed in the set- 
up instructions. 

b. Method of Instruction Giving - All instructions are oral end some demonstra- 
tion. Rriading is required of the client only when it is a requirement in the 
job area being sampled. 

c. Separation of Learning/Performance - Mosf of the work samples do not have 
a separate practice pciiod. Typically, the evaluator gives the instructions 
while providing a demonstration. The client attempts the task without a prior 
period of practice. There are no set criteria to be met prior to timing. 
Thus, there is minimal separation of learning from performance. 

d. Providing Assistance to Clients - Assistance can be given after the initial 
instruction period; but this results in lowering the client's final score. The 
manual contains detailed procedures for providing assistance and describes 
three levels of helping. Each level and each type are clearly defined. This 
emphasis of the analysis of the type of assistance is unique to the JEVS 
system. 

e. Repeating Work Samples - Readministration is not recommended because it 
invalidates results. 

6. Scoring and Norms 

a. Timing - A time clock is used to scamp the starting and stoppin,^ time for 
each work sample. A separate time stamp slip is used for each work sample. 

b. Timing Interval - The evaluator punches the time clock after instructions are 
given and the client punches the clock when the work sample is completed. 

c. Time Norms - Time results are raied on a three-point scale based on the 
number of minutes to completion. The scale is taken from percentile scores. 

d. Error Scoring - Most work samples use a random check of items that are 
compared to carefully defined scoiing criteria; many use photographs to 
illustrate quality standards. Assistance points are also incorporated into the 
error scoring procedures. 

e. Scoring Aids - Minimal use is made of scoring aids. 

f. Quality Norms - Quality is rated usii • a three-point scale based on the 
number of counted errors. 

g. Emphasis in Scoring - Time and quality are given equal weight. 

7. Observation of Clients 

a. Work Performance - Sixteen specific factors (e.g., size discrimination, form 
perception) and four more general factors (e.g., accuracy, neatness) are 
specified for the system; each work sample has certain factors listed that are 
to be observed. The system stresses the recording of accurate behavioral 
observations. 
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b. Work Behaviors - The system carefully lists and defines many work related 
behaviors that are to be carefully observed. For example, in writing observa- 
tions about communication, articulation, tone of voice and grammatical usage 
are to be noted. Some other behaviors are '^oopcrativeness with co-workers 
and supervisors, reaction to criticism, and frustration tolerance. 

c. Recording Systems - Many of the work performa'^.cc factors are rated on a 
three-point scale, with all ratings clearly defined and illustrated. 

d. Frequency of Observation - The system uses extensive observations. Observa- 
tion of defined work factors is required for each work sample; these are 
summarize*.d daily. 

8. Reporting 

a. Forms - Standardized forms are included for: reporting the results oi" each 
v'Ork sample, daily observation summary, feedback interview and a final 
report. 

b. Final Report Format - The well organized standardized format includes some 
ranking of work sample performance, recommended Work Groups and rationale, 
and extensive written comments on performanc and behavior. 

9. Utility 

a. Vocational Exploration - Client vocational exploration is seriously limited by 
two factors: (1) many of the work samples tend to be abstract, and (2) there 
is no orientation relating the work samples to jobs. 

b. Vocational Recommendations - The final report has r space for two Work 
Groups that are suggested for additional planning. The recommendations are 
related to the fourth edition of the DOT and the GOE and arc geared for 
both traininji and job placement. 

c. Counselor Utilization - The system and the final report are oriented toward 
t. e counselor; however, counselor familiarity with the DOT and GOE is r ccs- 
sary for optimal counselor use. 

10. Training Require d 

a. Training Required - Yes 

b. Training Available - Yes 

c. Duration - One week; usui>lly held in Philadelphia. Regional training is 
available under certain conditions. 

d. Follow-up - Not available. 

1 1 . Technical Con^>ideration 

a. Norm Base - The system was rcnoimed in 1975 on a total population of over 
1,100 cMeniS in 32 facilities throughout U.S. Time and quality norms are 
reported for the total sample as wel' as separate norms by sex, and for 
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different client groups, vocational rehabilitation., manpower. Goodwill, schools, 
and special schools for the mentally retarded. The norms are given in the 1- 
2-3 ratings only; no means, standard deviations or percentile cutoffs are 
given. Thus, the user has no idea of what the distribution is. Sample char- 
acteristics are not adequately described. 

b. Reality - No published data are available. 

c. Validity - Although the initial study of the system gave favorable evidence, 
results of studies done by the U.S. Department of Labor have not been re- 
leased to the public. Research by Nadolsky (1973) concludes that the system 
is valid for evaluation of immediate employment potential. There are no 
recent data available on validation. 

12. Reviewer^s Summary and Comments 

The JEVS System is a highly standardized and well integrated procedure for client 
evaluation based on 12 of the Work Groups of the GOE. The strongest points of 
the system are its stress upon careful observation and accurate recording of work 
behaviors and performance factors. The use of a trait and-f actor approach ties in 
well with the assessment of specific abilities. The major problems with the 
system appear to be the abstract nature of many of the work samples, which 
hinders vocational exploration and the lack of job information presented to the 
client. The system is best used when a thorough evaluation of the client's poten- 
tial is desired. 

13. Address 

Vocational Research Institute 

Jewish Employment and Vocational Service 

2100 Arch St. 6th Floor 

Philadelphia, Pennsylvania 19103 

14. Cost 

$9,980.00 includes all work samples, forms, and tuition for training one person in 
Philadelphia. The cost of transportation and living expenses for the person to be 
trained are not included in the price. 
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Prep Work Samples 



1. Development 

a. Sponsor - The work samples were developed by Prep, Inc, and are one of the 
four parts of the system originally called the Comprehensive Occupational 
Assessment and Training System (COATS), (i.e., Job Matching, Employability 
Attitudes, Work Samples, and Living Skills.) 

b. Target Group - The work samples were originally designed for use with 
special needs populations. Presently, the work samples are used in secondary, 
vocational education and alternative educational programs, manpower programs 
and corrections. 

c. Basis of the System - The work samples aie job simulations for aptitude 
assessment and career exploration. The 15 Career Clusters identified by the 
U.S. Office of Education were used for development of the Prep Work Sam- 
ples. These clusters were subdivided into 105 job families and their content 
is identified through job analysis and data contained in the third edition of 
the Dictionary of Occupational Titles. 

2. Organization 

a. Name and Number of Work Samples - Presently available work samples are: 

(1) Drafting - Sixteen tasks in architectural and technical drafting. 

(2) Clerical/Office - Filing systems, procedures and general reference 
materials. 

(3) Metal Construction - Lays out, cuts and files, solders, bevels, and 
threads. 

(4) Sales - Handles money and totals bills and credit forms. 

(5) Wood Construction - Try-square, hammering, materials calculation and 
sawing. 

(6) Food Preparation - Measures and mixes, calculates recipes and cooks 
short orders. 

(7) Medical Services - Determines TPR, determines blood pressure, keeps 
records and preforms urinalysis. 

(8) Travel Services - Clerical tasks centering around making airline reser- 
vations. 

(9) Barbering/Cosmetology - Combs and cuts hair, shampoos, and wig use 
and care. 

(10) Small Engine - Changes oil, cleans air filter; and tests magneto, spark 
plug and compression. 

(11) Masonry - Mixes mortar and lays bricks. 

(12) Electrical - Vires and mounts a junction box. 

(13) Police Science - Records information, completes reports and uses proper 
arrest procedures. 

(14) Electronics - Installs resistors, attaches wires, and tests circuits. 

(15) Automotive - Assembles tail light, attaches exhaust pipe and battery 
connector and rebuilds brake cylinder. 

(16) Commercial Art - Draws basic shapes and human figures, transfers 
letters and shapes and lays out an advertisement. 
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(17) Nutrition - Selects appropriate foods, checks nutrition and fills out 
forms. 

(18) Bookkeeping - Writes invoice and purchase order, operates calculator, 
corrects errors and transfers figures. 

(19) Fire Science - Records information, deals with public and selects proper 
equipment. 

(20) Extraction Technology - Measures solutions, calculates ppm, fixes 
oxygen, and cleans equipment. 

(21) Clothing and Textiles • Marks and cuts fabric, operates a sewing ma- 
chine, bastes, inspects and folds. 

(22) Real Estate - Lists and matches buyer's needs, completes a contract 
and determines costs. 

(23) Communication Services - Installs a speaker, mounts a transformer and 
tests equipment. 

(24) Refrigeration - cuts, reams and solders tubing; connects wires and 
installs a thermostat. 

(25) Computer Technology - Loads computer, enters data records, works with 
variables and writes a computer program. 

(26) Solar Technology - Assembles a solar panel. 

(27) Machine Trades - Sets up and operates a lathe; cuts; and performs shop 
calculations. 

b. Grouping of Work Samples - Each work sample is independent. Work samples 
are administered and scored individually. 

c. Manual - The general manual described the theoretical underpinnings and 
information applicable to all work samples. In addition, a specific manual is 
provided for each work sample. Detailed instructions, set up procedures, 
participant prerequisites, evaluation points, and other relevant information are 
contained in the specific manual. The general manual and the specific manu- 
als are contained in three-ring binders. 

Physical Aspects 

a. Packaging of Work Samples - Each of the 27 audiovisual work samples is 
packaged separately in portable containers and can be set up on a sturdy 
table or carrel. When not in use, the work samples can be easily stored. A 
single LaBelle projector may be used with several work samples. 

b. Durability - The work samples are durable. Containers are constructed of 3/4 
inch birch plywood and formica. Quality tools and equipment from manufac- 
tures such as Black and Decker, Stanley, and Craftsman are provided. The 
LaBelle audio-visual projector requires minimal maintenance and repaiis. 

c. Expendable Supplies - Each work sample uses consumables such as baking 
items, wood, printed circuit boards, and wigs. Enough consumables are pro- 
vided for assessing ten clients. Additional packages of consumables are 
available through Prep, Inc. The average cost of consumables per work 
sample is $6.50 per client, if purchased through the manufacturer. 

d. Replacement - With the exception of the Self Rating Forms and Individual 
Report Forms that must be ordered from Prep, Inc., all expendable supplies 
can be purchased locally. 
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e. Computer Requirements - Not applicable. 



Work Evaluation Process 

a. Preliminary Screening - No preliminary Screening is required. However, the 
evaluator should review the prerequisites for each work sample to avoid 
unnecessary frustration on the part of the client. 

b. Sequence of Work Sample Administration - There is no predetermined ad- 
ministration sequence. Both the order and number of work samples admin- 
istered are determined by the client and evaluator. 

c. Client Involvement - The client completes a Self Rating Form based on inter- 
est, perceived difficulty and level of performance throughout the assessment. 
At the conclusion of the work sample administration, the evaluator transfers 
the client's self-ratings to the Individual Record Form, which includes the 
evaluator's observations. The results of the client's and the evaluator's 
ratings are reviewed and the client is given a copy of the Individual Report 
Form. 

d. Evaluation Setting - The majority of the equipment replicates industry. 
However, since assessment usually takes place in a room w-th tables, carrels, 
and audiovisual projectors, a classroom atmosphere prevails. 

e. Time to Complete Entire System - Because the average work sample is com- 
pleted in two hours, all 27 work samples would take an estimated 54 hours. 
Average work sample times range from 38 minutes to four hours and 12 
minutes. However, in actual use, this many work samples would not be 
administered. The average client completes from three to five work samples 
in from six to ten hours. 

Administration 

a. Procedures - The materials required, layout procedures and evaluation points 
are clearly described. 

b. Method of Instruction Giving - Instructions are presented with an audiovisual 
LaBelle format. This format uses an eight track audio tape synchronized with 
a 16mm filmstrip. Automatic stop pulses are programmed throughout each 
work sample allowing the client to complete each step and work at his/her 
own pace. 

c. Separation of Learning/Performance - There is minimal separation of learn- 
ing/performance. Most client instructions do not require a criteria to be 
reached before proceeding from the practice period to the performance period. 
For each task on which the client will be assessed, a demonstration of the 
task is always given prior to the evaluation point. Many times the client is 
afforded the opportunity to practice the task first. 

d. Providing Assistance to the Client - If a client is having difficulty with a 
task, the manual recommends that the evaluator record this difficulty. If the 
client cannot complete this task, the work should be evaluated as less than 
acceptable. The evaluator may assist the client with a task if the remainder 
of the work sample is based upon the successful completion of a task. 



c. Repeating Work Samples - Generally, work samples are not readministered. 
Scoring and Norms 

a. Timing - Time is usually recorded, but has no bearing on the evaluation 
except as an indication of severely limited ability. 

b. Timing Intervals - Since timing is not critical to scoring, the client is timed 
during the entire duration of the work sample. Appropriate stopping points 
are indicated to allow for short class periods or interruptions. 

c. Time Norms - The evaluator recoras the total work time on the Individual 
Report Form. Separate time norms are not used. 

d. Error Scoring - At certain evaluation points during the work sample, results 
are checked against carefully defined scoring criteria. Each performance is 
evaluated separately on a five point acceptability scale and an overall rating 
(i.e., high, medium, and low) is assigned. A rating is also assigned for be- 
havioral observations. 

e. Scoring Aids - Some scoring aids are used. For example, overlays are used 
for the completed drawings in the Drafting Work Sample and response ki ys 
are supplies with each work sample requiring written work. 

f. Quality Norms - The acceptability ratings are based on industrial standards. 
Performances must be completed with certain tolerances in order to receive 
acceptable ratings. Exact tolerances are described for each work sample in 
the specific manuals. 

g. Emphasis in Scoring - The system emphasizes work performance and work 
behavior ratings. The client's performance time and self-rating are also taken 
into consideration. 

Observation of Clients 

a. Work Performance - Work performance factors are not listed. 

b. Work Behaviors - The Individual Report Form lists eight work behaviors 
which are to be observed and recorded as part of the evalu?»tion. They are: 
relationship to authority, relationship to co-workers, tolerance for frustration, 
acceptance of criticism, concern for property, work efficiency, reliability, and 
appropriateness of appearance. 

c. Recording System - The Individual Report From describ.^s high, medium and 
low ratings. A rating is assigned for each work performance and work be- 
havior, and an average is taken for an overall rating. 

d. Frequency of Observation - No predetermined schedule exists. Frequency will 
vary according to the number of clients being evaluated as well as the fre- 
quency of work performance evaluations. 
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8. 



Reporting 



a. Forms • Two forms are used: the Self-Rating Form and the Individual Report 
Form. The Self-Rating Form is used by the client to record his/her interest 
in the job families represented by the work sample both before and after 
administration. In addition, the client records the level of difficulty or each 
performance and his/her own perception of how well the task was performed. 
The Individual Report Form is completed by the evaluator and includes work 
performance ratings, work behavior, work rate, and summary of the client*s 
Self-Rating. 

b. Final Report Format - The Individual Report Form becomes part of the final 
report. Based on the data entered, a narrative summary of the results is 
written at the top of the form. Since the form has its own carbon copy, the 
client can have a copy of the assessment results. On the reverse side of the 
form is a preprinted description of the work sample and specific job rv^lated 
information. Upon completion of all work samples administered, a narrative 
composite of all results is usually prepared. 

9. Utility 

a. Vocational Exploration - The client has a wealth of career information avail- 
able in the work samples. Occupational information is presented at the 
beginning of the audiovisual cartridge. All tasks are depicted in the work 
environment. 

b. Vocational Recommendations - Based upon the results of the work sample 
evaluation and the preprinted data on the reverse of the report form, the 
counselor could make job, training, or further exploration recommendations 
using the DOT classifications provided. 

c. Counselor Utilization - The report forms generated by completion of each 
work sample can be used by counselors and clients alike. The front of the 
forms contains the evaluation data and the reverse preprinted side contains a 
description of the work sample and job related information. 

10. Training in the System 

a. Training Required - Although not required, training is strongly recommended. 

b. Training Available - Training is available and can be presented either at Prep, 
Inc. or on site. 

c. Duration - One full day of work sample training is usually suf f;cient. 

d. Follow-up - Customers may contact Prep directly for technical assistance or 
to arrange follow-up training. 

11. Technical Considerations 

a. Norm Base - Work sample norms reflect the criteria of acceptability used as 
industrial quality standards. Client*s work is judged against stated precstab- 
lished levels. Time norms are not used. 
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b. Reliability - No information available. 

c. Validity - Work sample validity is based solely on content validity, consisting 
of job analysis and the DOT. The methods are logical and result in a group 
of tasks that represent occupations within a particular job grouping. No 
other data on validation are available. 

12. Reviewer's Summary and Comments 

The Prep Work Samples are well-designed modules useful with a wide variety of 
disabled persons. The use of audiovisual instruction methods permits the presen- 
tation of occupational information as well as close monitoring of client progress. 
The manual clearly outlines the tasks comprising each work sample and how each 
task is assessed. One of the better features of the system is that each sample 
was designed around tasks common to several related occupations, instead of 
attempting to duplicate all the tasks present in one job. The construction and 
tools used in the Prep Work Samples give the impression of long use if routine 
maintenance is performed. 

There are two potential problems in using these work samples with a rehabilita- 
tion population: (1) the use of audiovisual format can present problems for 
persons with hearing, visual and/or learning handicaps and (2) although designed 
in part for special needs students, the system does not seem appropriate for some 
lower functioning mentally retarded persons. 

In spite of many genuinely good points, the system fails in the reliability and 
validity. There is simply no information available on the reliability of the system. 
Validity is based solely on a content analysis with the DOT and job analyses. 
Other types of validation are definitely needed. 

13. Address 
Prep, Inc. 

1007 Whitehead Road Extension 
Trenton, New Jersey 08638 

14. Cost 

Work sample prices range from $480.00 (Fire Science and Police Science) to 
$4,250.00 (Computer Technology); the average price is $1,343.70. Each work 
sample comes with tools, cartridges, specific manual, storage cube, and ail expen- 
dable materials for assessing ten clients. The cost of the Self-Rating Form and 
Individual Report Form is $12.50 per set of ten. 

15. References 

Micali, J. J. (1977). Comprehension levels of the COATS and Singer Vocational 
Evaluation Systems: Implications for use with the retarded. Vocational Evalu- 
ation and Work Adjustment Bulletin. iO(3), 28-31. 

Prep, Inc. (1976). Preliminary "useability" results of a field study to determine 
the appropriateness and effectiveness of COATS with special education popula- 
tions. Prep Research Reports . 1(15). 
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System. Vocational Evaluation and Work Adjustment Bulletin , £(3), 39-45. 

Texas Educational Agency Project VOC-AIM. (1982). Review of commercial work 
sample systems. Vocational assessment of students with special needs: An 
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Pre-Vocational Readiness Battery 



(Valpar 17) 



1. Development 

a. Sponsor - Valpar was developed by Valpar International. 

b. Target Group - The battery is aimed at assessing the functional skills of 
mentally retarded persons. 

c. Basis of the System - The manual contains no discussion on the basis of the 
system. 

2. Organization 

a. Name and Number of Work Samples - The system contains five areas, each of 
which has several separate subtests: 

(1) Development Assessment - Contains four parts which are ^'simple, func- 
tional, non-medical measures of physical and mental abilities": (a) 
Patterning/Color Discrimination Manipulation, (b) Manual Coordination, 
(c) Work Range/Dynamic Strength/Walking and (d) Matching/Vocational 
Knowledge/Measurement. 

(2) Workshop Evaluation - A simulated assembly process during which three 
clients use a three step assembly process. A fourth person (either client 
or evaluator) acts as an inspector. 

(3) Vocational Interest Screening - A sound/slide interest assessment in 
which the client compares two jobs. There are six area scores: social 
service, sales, machine operation, office work/clerical, physical sciences, 
and outdoor. 

(4) Social/Interpersonal Skills - This consists of a two page form containing 
descriptions of commonly found barriers to employment. Four major 
areas are covered: (a) personal skills, (b) socialization, (c) aggravating 
behaviors, and (d) Arork related skills. 

(5) Independent Living Skills - An assessment of: (a) transportation, (b) 
money handling, (c) grooming, and (d) living environments. The trans- 
portation and money handling areas contain three levels. Simulation and 
gaming techniques are used heavily in this area. 

b. Grouping of Work Samples - The subtests arc grouped according to the five 
areas given above. 

c. There is an overall manual and a separate manual for each arc a, plus a sixth 
manual containing norms. Each manual is well organized and contains most 
setup, administration, and scoring instructions. The scoring instruction ex- 
amples are unusually detailed. 
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3. Physical Aspects 



a. Packaging of the Work Samples - Equipment for each of the five areas is 
packaged separately. 

b. Durability - As with other Valpar work samples, all equipment is very durable. 
Construction is fiberboard laminated with formica. 

c. Expendable Supplies - Aside from the numerous forms and recording sheets, 
the system requires no expendable supplies. 

d. Replacement - Forms can be ordered from the developer or may be repro- 
duced locally. 

e. Computer Requirements - Not applicable 

4. Work Evaluation Process 

a. Preliminary Screening - Subtest One: Developmental Assessment is used as a 
preliminary screening to determine the evaluec's general physical strength, 
mobility, and instruction following skills. 

b. Sequence of Work Sample Administration - information presented by the 
developer states that the five sections can be given in any order. However, 
because the method of instruction giving for the rest of Valpar #17 is at 
least partially determined during the Development Assessment section, this 
part should be given first. 

c. Client Involvement - While the degree of client involvement with the evalu- 
ator varies with the section, in general there is a considerable degree of 
client-evaluator contact. Most of the tasks are administered individually. 
The manuals do not contain any discussion on procedures for feedback and 
for sharing the result with the client cither during or after completion of 
the five arcs. 

d. Evaluation Setting - The setting is not specified. However, the use of the 
various sections implies that a formal testing situation is created. 

c. Time to Complete the Entire System - While the time varies with the popula- 
tion tested, the general manual estimates 5 and a half hours for the entire 
battery. 

5. Administration 

a. Procedures - Administration procedures layout, materials needed, and general 
instructions are clearly given in the manuals. 

b. Method of Instruction Giving - Instructions are given using a variety of 
methods. During the administration of the Development Assessment section, 
the cvaluator is first to determine at which of three possible levels the client 
functions: (1) verbal, (2) verbal plus demonstration, and (3) verbal plus 
demonstration with a sample to follow. The appropriate level is used 
throughout the remainder of the Valpar #17. The Vocational Interest Screen- 
ing uses a slide/cassette instruction method. Independent Living Skills uses a 
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combination of gaming and comparing pictures with accompanying verbal 
instructions. 

c. Separation of Learning/Performance - In the Workshop Evaluation part, some 
separation of learning from performance occurs; the client is corrected if 
he/sae makes a mistake during the instruction period. However, there are no 
set criteria. This section of the outline is not appropriate for the other four 
parts. Each of these areas is an assessment of knowledge and of the ability 
to learn. 

d. Providing Assistance to the Client - The evaluator is to make certain that the 
client can perform the task or activity. In most parts, extra assistance would 
not interfere with the results. However, this area is one of the few areas 
not clearly covered in the manuals. 

e. Repeating Work Samples - Readministration is strongly recommended when it 
would facilitate either program evaluation or documentation of changes in 
client skills over a period of time for the purpose of adjustment areas of 
emphasis in training. 

Scoring and Norms 

a. Time Scores - Except for one task in the Development Assessment Unit, no 
time scores are recorded; all parts are untimed except the Workshop Evalua- 
tion. In this part, the number of units assembled in 12 minutes is the score. 

b. Timing Interval - In the Workshop Evaluation, the 12 minute interval begins 
after the clients have understood the instructions and have practiced. 

c. Time Norms - No time norms are used, except for the one subtest noted 
above. 

d. Error Scoring - All parts arc scored on the number of correct responses, 
except Interpersonal-Social Skills. In the Developmental Assessment Unit, 
most tasks are scored by giving either 4, 2, 1 or 0 points; many of the 
physical capacity evaluations record the number of pounds lifted or moved. 
The Vocational Interest Screening uses the number of choices in the six work 
areas. The Social-Interpersonal Skills uses a negative score in which the 
more skills and/or behaviors that are lacking, the greater the scores in each 
of the four areas. Finally, the numerous scored activities in the Independent 
Living Skills assign one or more points to each correct response. 

c. Scoring Aids - No scoring aids are used. 

f. Quality Norms - The only "quality norms," as this term is used in work 
sample scoring, are those for the Workshop Evaluation. The other four 
sections use norms based on the number of total points. 

g. Emphasis in Scoring - The scoring emphasis is on the number of "correct" or 
appropriate responses. 
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7. 



Observation of Clients 



a. Work Performance - None of the five sections identify any specific work 
performance factors. 

b. Work Behaviors - The Interpersonal-Social Skills area contains a section on 
work related skills. Some of the specific items are: (1) safety, (2) prompt- 
ness, (3) following directions, and (4) work completion. Some of these skills 
are not definei? in behavior terms. The Workshop Evaluation contains the 
following four items: (I) corrected work, (2) work backed up, (3) on task, 
and (4) work cohesively with others. Each is rated on a three-point scale. 
Other areas include space on forms for writing in general comments and 
observations. 

c. Recording System - For the Interpersonal-Social Skills, "^behaviors" are rated 
using 0, 2, or 4 as a weight. The Workshop Evaluation uses a three-point 
scale of "never," "sometimes," and "consistently." 

d. Frequency of Observation - This point is really only applicable to the Work- 
shop Evaluation section; the manual does not specify the frequency of obser- 
vation. 

8. Reporting 

a. Forms - Each of the five areas uses standardized, well-designed forms for 
recording responses and for scoring. Some of the fo^ois include black and 
white reproductions of the stimulus slides and the transportation gaming 
exercise. 

b. Final Report Format - This is unspecified due to the wide variety of settings 
and applications for which the work sample was designed. However, the 
Individual Exit Profile provides a summary of all scores in a manner which 
transfers to individual education plans used in an educational setting. This 
same form provides a functional summary around which the narrative section 
of most reporting formats can be organized. 

9. Utility 

a. Vocational Exploration - Two parts of the Valpar #17 offer some direct 
vocational exploration: Vocational Interest Screening and Workshop Evalua- 
tion. The Interest part allows for some exploration and provides some oc- 
cupational information. The Workshop Evaluation, as a simulated assembly 
task, could give the client some concept of production line work. 

b. Vocational Recommendations - Each of the sections provides data -hat can be 
used to provide vocational recommendations. The specific recommendations 
would be based upon the final reporting format used. 

c. Counselor Utilization - The battery is designed specifically to facilitate 
counseling and/or training after assessment. The scoring format specifies 
goals and potentials by providing counselor insight into relative strengths and 
weaknesses. It also provides score sheets with pictorial representations of 
the work performed in that task to remind both the counselor and the eval- 
uee of the activity and performance in each area. Each subtest also provides 
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both a possible means of remediation or training and a format for reassess- 
ment to gauge improvements over time. 

10. Training in the System 

a. Training Required - No training is required. 

b. Training Available - Formal training is available. 

c. Duration - At least one day or more, this depends upon evaluator needs. 

d. Follow-up - As requested by the user. 

11. Technical Considerations 

a. Norm Base - A separate norms manual contains what the developers call 
"research norms" on 10 different groups (e.g., competitive employment, 
sheltered workshop, activity center, and homebound employment). The raw 
scores for each subtest are converted to a single percentile score. (The 
manual contains no information on how this percentile score was developed.) 
Over the percentile score on each table are three normal distribution curves, 
which represent three combinations of the norming groups. The user can 
apparently roughly determine where on these curves a person falls. Because 
there are no means, standard deviations, sample sizes, or descriptions of the 
samples available, it is impossible for this reviewer to make any comment 
regarding the norms. 

b. Reliability - No data presently available. 

c. Validity - No data presently available. 

12. Reviewer's Summary and Comments 

Valpar #17 is intended to be an assessment of the variables that must be con- 
sidered when assessing a mentally retarded person's interests, vocational skills, 
and social maturity. The system is designed to be used by a person who is not 
trained in psychology, medicine, or occupational therapy. The system is well 
designed, attractive, and novel in many ways. The use of audiovisual and gaming 
materials will make it attractive to clients as well as evaluators. Data collection 
forms are unusually well designed. The major problems are in the technical areas. 
The manuals contain no background as to why certain components were selected, 
no relationship to previous work done in this field. No data are given on relia- 
bility and validity; there is not even a statement on these two factors. The rorm 
data are impossible to interpret w.**hout additional information. In summary, this 
is a very attractive assessment device, but much more needs to be known about 
it. 

13. Address 

Valpar International 
P.O. Box 5767 

Tucson, Arizona 85703-5767 
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14. Cost 



Pre-Vocational Readiness Battery. $4,595.00 

Subtest - 1 $895.00 

Subtest - 2 $995.00 

Subtest - 3 $350.00 

Subtest - 4 $250.00 

Subtest - 5 $2,365.00 



15. References 

Botterbusch, K. F. (1980). Pre-vocational readiness battery. In A. Sax (Ed.), 
Innovations in vocational evaluation and work adjustment. Vocational Evalu- 
ation and Work Adjustment Bulletin. 11(1), 26-28. 

Nichclsun, J. R., Nailen, P. M., & Tobaben-Wyssmann, S. (1984). Valpar #17 Pre- 
Vocational Readiness Battery: A question of norms. Vocational Evaluation and 
Work Sample Bulletin, 17(3), 67-70. 
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Skills Assessment Module 



(SAM) 



1. Development 



a. Sponsor - SAM was developed by Piney Mountain Press, Cleveland, Georgia. 
The manual was written by Michele Rosinek. 

b. Target Group - SAM "is designed to assess students (13-18 yrs.) identified by 
the school system as mildly handicapped (learning disabled, emotionally dis- 
turbed, mildly retarded and/or disadvantaged (economically and/or education- 
ally). No screening is required beyond this point." 

c. Basis of the System - SAM is designed to access 24 behaviors (e.g., appear- 
ance and endurance) and certain general "aptitudes." The following sources 
were the basis of the system: U.S., Department of Labor Aptitudes, D-Map 
(i.e., Dunn Model for Assessment and Placement), Guide for Occupational 
Exploration, the Occupational Aptitudes Patterns of the USES General Ap- 
titude Test Battery, and competency requirements in vocational training 
programs. 



2. Organization 



a. Name and Number of Work Samples - The testing part of the system contains 
three paper-and-pencil tests and 12 work samples: 



Paper-and-Pencil tests: 



(1) Revised Beta Examination (Second Edition) 

(2) Leaning Styles Inventory developed by Piney Mountain Press 

(3) Oral Directions Test from the Personnel Tests for Industry 



The 12 short work samples are as follows: 



(1) Mail Sort - Sorts 100 postcards by zip code. 

(2) Alphabetizing - Files 50 postcards. 

(3) Etch-A-Sketch Maze - Follows a maze overlayed on an Etch-A-Sketch. 

(4) Payroll Computation - Computes a payroll for 9 persons. 

(5) Patient Information Memo ■ Recopies information onto a form. 

(6) Small Parts A - Inserts washers in slots with tweezers. 
Small Parts B - Screws small screws into a metal block. 

(7) Ruler Reading - Measures predetermined lines and draws lines to spe- 
cific lengths. 

(S) Pipe As^^embly - Assembles cast iron pipes and fittings. 

(9) O Rings - Sorts 12 different sizes of O rings. 

(10) Block Design - Constructs a design according to a pattern. 

(11) Color Sort - Sorts 50 cards by seven different colors. 

(12) Circuit Board - Constructs a circuit following a diagram. 



b. Grouping of Work Samples - SAM is designed to be flexible. Each work 
sample is independent and can be administered in any order or in as many 
settings as desirable. 
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c. Manual - The major sections of the manual are in logical sequence and flow 
from one section to the next. The manual defines all of the ap.-tudes and 
behaviors that the system claims it measures, general administration guide- 
lines, instructions for each work sample and sections on scoring. Instructions 
for each work sample include the materials needed, purpose, interpretation 
and directions for administering and scoring. The major problem with the 
manual is the administration instructions for each work sampl \ These are 
often vague and imprecise. Valuable information is sometimes missing; for 
example, there arc considerable gaps in the standardized instructions and 
demonstrations to the client. 

Physical Aspects 

a. Packaging of the Work Samples - The entire system is contained in a wooden 
box about 2x3x11/2 feet. Within this box there is a compaitment for 
each work sample. Each sample is packaged separate with all its components. 

b. Durability - Most of the samples are made of wood and plexiglass. With the 
exception of the file cards and other paper parts, the system appears to be 
fairly durable. 

c. Expendable Supplies - In addition to the paper-and-pencil tests, forms for the 
Ruler Reading, Patient Information and Payroll Computation are required for 
each student. A separate data entry form and paper for the printer are also 
needed. 

d. Replacement - The manual contains a complete parts list; all parts can be 
ordered from Piney Mountain Press. 

e. Computer Requirements - SAM contains one floppy disk to score and print 
student results. A second disk is available to develop a local scoring and 
task matrix program (see section on validity). This software is available for 
IBM-PCs and Apple II, He and lie. Each requires a computer, monitor, one 
disk drive and printer. 

Vocational Evaluation Process 

a. Preliminary Screening - No screening is required. 

b. Sequence of Work Sample Administration - The three paper-and-pencil tests 
are always administered first. The work samples can be given in any order 
and in as many settings as desired. 

c. Client Involvement - The manual does not contain any specific procedures for 
sharing results with the client. Apparently the client is not involved in 
planning the SAM evaluation. For each module the student is given an 
orientation regarding the purpose and expectations of the tasks; there is little 
client involvement beyond this. 

d. Evaluation Setting - Because SAM is intended for a school population, it is 
assumed that evaluation is done in a classroom atmosphere. 

e. Time to Complete Entire System - The manual states that administration takes 
from 1 1/2 to 2 1/2 hours. 
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5. Administration 



a. Procedures - The manual section on each module includes the purpose, mate- 
rials needed, a description of the work sample "activity," verbal cue, inter- 
pretation and scoring. In spite of these headings, there are administrative 
problems. The "activity" and "verbal cue" sections are often vague and 
require several readings before they can be fully understood. Verbal and 
demonstration instructions should be clearly given. 

b. Method of Instruction Giving - All directions are given orally and by demon- 
stration. The teacher is to give the student ample time to acquire the neces- 
sary skills for each module prior to timing. 

c. Separation of Learning/Performance - Before each work sample the client is 
to be given an orientation regarding the purpose and expectations of the tests 
given. The manual states: "Always allow for practice when indicated and 
make sure the participant acquires the required skills and concepts needed to 
perform the task before testing." Thus, there appears to be clear separation 
between learning and performance. 

d. Providing Assistance to the Client - The manual clearly states that "no assis- 
tance may be given during the performance phase." 

e. Repeating Work Samples - "After the initial administration of all the samples, 
the participant may be allowed to practice on specific samples to determine if 
he or she can score close to the MTM time...Simply readminister sample in 
the same manner as indicated on individual instruction sheets and keep a 
record of how many practice trials it takes to reach a competitive time." 

6. Scoring and Norms 

a. Timing - Timing is started after the piactice session and continues either 
until the task is completed or until the maximum time for each work sample 
is reached. The evaluator times the client. 

b. Timing Interval - Once the client has demonstrated adequate skill through the 
training (i.e., practice) session, the evaluator instructs the client to begin. 
Timing begins at this point. Each work sample has time limits and the client 
is instructed to stop if he/she has not completed the task by that time. 

c. Time Norms - Time and error data can either be scored by hand or fed into a 
computer program which determines time and error norms. Time scores are 
presented in percentiles based on student ncrms and MTM-1 industrial norm 
scores. 

d. Error Scoring - The number of individual errors are counted and entered into 
the scoring program. Errors for some of the samples (i.e.. Patient Informa- 
tion Memo, Pipe Assembly, Block Design and Circuit Board) are not clearly 
defined. 

e. Scoring Aids - Some of the work samples, such as the Mail Sort and Alpha- 
betizing, use scoring aids. Other work samples have illustrations showing the 
correctly completed task. Pages with correct responses are supplied with one 
work sample requiring written responses. 
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f. Quality Norms - Student error norms are available and are reported in per- 
centiles. 



g. Emphasis in Scoring - The SAM manual clearly states that time and errors 
are of equal importance. 

7. Observation of Clients 

a. Work Performance - No work performance factors are used in the SAM. 

b. Work Behaviors - The following eight work behaviors are rated on a five 
point scale: appearance, communication skills, conforms to rules, endurance, 
initiative, interpersonal traits, reactions to assigned tasks and safety con- 
sciousness. All are generally defined in the glossary section of the manual; 
behavioral definitions are not used and no specific behaviors are defined for 
each point on the five point scale. Specific behaviors are not related to 
specific work samples. 

c. Recording System - The Data Form contains a scale for rating behaviors, 
ranging from 1 ("superior") to 5 ("needs improvement"). The reader should 
note that no schedule or system of observations is required, nor is the evalu- 
ator required lo reference or justify these very global ratings. Thus, only a 
global rating is recorded. 

d. Frequency of Observation - Observations are to be made throughout the 
entire assessment period. 

8. Reporting 

a. Forms - The Data Form is the one major form used. It is divided into three 
sections: (1) student identification, test scores and ratings for the verbal, 
numerical and GED levels; (2) work sample time start and stop, number of 
errors and norm scores; and (3) a five-point "Performance Graph" and the 
rating scale for the behaviors. 

b. Final Report Format - The written final report uses the Data Form, men- 
tioned above. The computerized scoring system also generates a form con- 
taining percentile, MTM results and graphic presentations of the scores. A 
second computerized report compares student performance with traits needed 
for success in various vocational training courses within the client's school. 
There are no provisions for a narrative report. 

9. Utility 

a. Vocational Exploration - Due to the formal testing situation, the simplicity of 
the work samples, and the lack of client orientation relating work sample 
tasks to job'j, the system provides very little vocational exploration. 

b. Vocational Recom^nendations - Recommendations can be made for specific 
training programs within a school system for the student. These training 
recommendations are the most important system output. 
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c. Counselor Utilization - The system is designed to inform the counselor and 
student on "the student's level of performance as it relates to motor, cogni- 
tive, and affective demands inherent in an instructors' various training pro- 
grams." If accurate, this information would be a very realistic tool for 
student choice. 

10. Training 

a. Training Required - Training is not required prior to sale. 

b. Training Available - Training is available 

c. Duration - Duration is dependent on the needs of the evaluator and the 
number of persons to be trained. Time ranges from two hours to one day. 

d. Follow-up - Technical assistance is available after purchase through the 
publisher. 

11. Technical Considerat i ons 

a. Norm Base - SAM has both predetermined time study and psychometric norms. 
The predetermined norms were developed with MTM-1; the technical manual 
contains the 100% or industrial normal times. There are no conversion tables. 
Psychometric norms were developed on three samples consisting of "average 
students," "handicapped students" and "disadvantaged students" at "eight data 
collection sites in urban, suburban, and rural secondary and post secondary 
schools." Age, sex, education level and area of residence (i.e., urban, subur- 
ban and rural) are given for each sample. All norms are from Georgia. 
Specific sample sites, procedures and methods were not given. Thus, the user 
has limited information to interpret what a particular score means. There are 
two other problems with the norms: the largest sample consists of only 121 
students, and the error score distributions were skewed. 

b. Reliability - The technical manual presents a single test-retest study over a 
three to five day period using Pearson's r. All of the correlations arc over 
.80. These findings are very encouraging. 

c. Validity - A job-task matrix based on U.S. Department of Labor data was 
developed. This contains ratings of the importance of the factors as related 
to various school training programs. The user can modify this matrix for local 
use. The manual makes a serious error by assuming that the tasks are valid 
because they have MTM norms on the work samples (Botterbusch, 1981). 

Validity for the SAM must be locally developed. The teacher has vocational 
teachers rate the degree to which each of the iraits (e.g., verbal aptitude, 
appearance, manual dexterity) measured by SAM are required for success 
within that teacher's vocational program. This process is computerized and 
the evaluator can enter new data as needed. This emphasis on locally devel- 
oped validity is seen by this reviewer as a positive step. 

12. Reviewer's Summary and Comments 

SAM is designed to give the teacher/evaluator a comprehensive, flexible and 
affordable assessment tool. It is intended to evaluate aptitudes, work behaviors, 
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learning styles and cognitive performances of students who are interested in 
entering the vari'. us training programs within a school system. The system can 
be administered in a i datively short time, h also has the advantage of permit- 
ting the evaluator to rcadminis^cr work samples as a student training experience. 
Finally, the concept oi' relating assessment results to vocational training programs 
within the student's school is very practical and very useful. 

SANTs problems are many and most of these certer on the technical aspects of 
the system. The 24 traits measured ure not c?refuily defined and there is no 
empirical evidence relating these traits to specific work samples. To compound 
this problem, there are no factor analysis studies to justify these traits. 

The error scores and norms are badly ,*ewed; this suggests that the samples are 
too easy for most of the students. Th(; reliability study is not described well 
enough to permit the evaluator to judge thv^se critical aspects of the system. 

The major problcni iz with vaiiaity. Although the idea of developing local crite- 
rion for vocational training programs is very sound, the procedures given in the 
manual for developing such a system are totally inadequate. For example, there 
are no data collection forms, nc procedures for relating the trait definitions to 
training requirements, no way to determine teacher reliability in ratings and, most 
important, no method for determining if the teacher ratings are valid. Thus, the 
system takes teacher ratings at face value. When this problem is combined with 
the lack of rigorous trait definition, the results are an instrument with unknown 
validity. 

13. Address 

Piney Mountain Press, Inc. 
P.O. Box 333 

Cleveland, Georgia 30528 

14. Cost 

Skills Assessment Module (includes: Manual, Revised Beta, Learning Styles Inven- 
tory, 12 work samples, scoring and disk for Apple or IBM computers and on-site 
training) $1,695.00 

Local Norm Development System (includes I disk and documentation) . $195.00 

15. References 

None presently available. 



118 

118 



System for Assessment and Group Evaluation 



(SAGE) 



1. Development 

a. Sponsor - SAGE was originally developed by Schabacher and Associates and 
Creative Development Associates, Inc.; rights are presently owned by the 
Train-Ease Corporation. 

b. Target Group - The system is currently being used by junior and senior high 
school students and in post-secondary education. According to the developer, 
it is especially useful with disadvantaged and some handicapped persons. 
Successful use has been reported with worker compensation and head trauma 
cases. It can also be used in private industry for hiring and re-training. 

c. Basis of the System - The fourth edition of the DOT, the Guide for Occupa- 
tional Exploration, the Worker Trait Group Guide (Appalachia Educational 
Laboratory, 1978a) and Career Information System Guide (Appalachia Educa- 
tional Laboratory, 1978b) formed the basis of the system. In general, SAGE 
was designed to assess for the traits in the Worker Trait Profile. 

2. Organization 

a. Name and Number of Work Samples - The system consists of five different 
components (i.e., GOE Interests, Aptitudes, GED and Temperaments) that when 
used together are intended to give the evaluator a complete pictuie of the 
client on most of the variables contained in the Worker Trait Profile^. The 
five components are defined below: 

(1) Vocational Interest Inventory (VII) - This untimed paper-and-pencil 
inventory measures the client's interests in the 12 interest areas taken 
from the Guide for Occupational Exploration. ^^ For each of the 152 
items, the client makes two possible responses: (1) circles one response 
if he/she likes the activity and (2) circles another n*.sponse if he/she 
has done the activity. Each item uses the following format: 

Put parts on a car 

Get people to like your ideas 

Make beds in a hospital 

(2) Vocational Aptitude Battery (VAB) - The VAB uses a combination of 
paper-and-pencil tests and isolated trait work samples to measure 11 
aptitudes defined by the U.S. Department of Labor: 

General - A multiple choice paper-and-pencil test with verbal, 
arithmetic reasoning and spatial items. 



^ The only two parts of the Worker Trait Profile not measured by SAGE are 
Physical Demands and Environmental Conditions. 

For a list of the 12 areas, see "interests" in the Glossary. 
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Verbal - A papcr-and-pencil test using a stimulus word with four 
sets of response words. The person chooses the response word 
that is the same or opposite in meaning. 

Numerical - A paper-and-pencil test containing subtests in compu- 
tational items and "reading problems." 

Spatial - A performance requiring the client to reproduce patterns 
of varied sized gears on a console using a photog'*iph as a stimu- 
lus. 

Form Perception - A multiple choice test with several colored pages 
of tools and materials; the client finds the tool that matches the 
stimulus photograph. 

Clerical Perception - Consists of four subtests, two multiple choice 
and two performance; a booklet is checked to determine if names 
and numbers are the same; cards are sorted by numerical and 
alphabetical order. 

Motor Coordination - "A manipulative test that requires clients to 
perceive a light flashing on a large electronic control board and 
within a half second period of time push or hit a button under the 
light with either hand." One hundred lights flash per minute; this 
is self-timing and self-scoring. 

Finger Dexterity - Small specifically modified compression unions 
are assembled on 3/16 inch pins; no tool are used. 

Manual Dexterity - Large conduit fittings are assembled on a 
special console. No tools are used; the client must use his/her 
entire hands and wrists. 

Eye-Hand-Foot Coordination - Using a test similar to that used for 
World War II and Korean War pilot selection, the client uses a 
stick and foot pedals to match three sets of lights showing on a 
console. 

Color Discrimination - A multiple choice test consisting of several 
pages of standard color samples. The client must find the colored 
dot that matches the stimulus. 

(3) Cognitive and Conceptual Abilities Test (C-CAT) - This multiple choice 
paper-and-pencil test measures six levels of General Educational Devel- 
opment (GED). There are separate subtests for Reasoning, Mathematics, 
and Language. Results are converted to scale scores from one to six. 

(4) Assessment of Work Attitudes (AWA) - An untimed 30 item scale cover- 
ing 20 common work attitude categories, such as workmanship, deferred 
gratification, and persistence as they relate to a specific work related 
situation. A typical item is as follows: 
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How many people believe in always being on time for work? 



a. 1 out of 5 

b. 2 out of 5 

c. 3 out of 5 

d. 4 out of 5 

(5) Temperament Factor A"»scssmcnt Unit - This 90 item untimcd true and 
false test measures the ten U.S. Department of Labor Temperaments^^. 

b. Grouping of Work Samples - The tests, attitude scales, etc. are grouped into 
the five areas outlined above. 

c. Manual - The loose-leaf binder contains all system details. There is a sepa- 
rate section for each test. Each contains general information, specific in- 
structions for clients and administrators, and manual and machine scoring. 
Other sections of the manual give copies of forms, provide scoring examples, 
and offer some information on interpretation. In general, the manual is well 
written and easy to follow. 

3. Physical Aspects 

a. Packaging of the Work Samples - Each test apparatus is packaged indepen- 
dently. 

b. Durability - While no durability data are available, the plastic cases give the 
impression of being fairly durable. 

c. Expendable Supplies - There are no consumable supplies except for forms and 
answer sheets; these may be produced locally. 

d. Replacement - An 800 telephone number is supplied for securing replacements 
or service for the test apparatus. To maintain standardization, users should 
check; with SAGE before using local parts. 

e. Computer Requirenicnts - The SAGE system has an optional machine scoring 
system called "Auto-Score" that includes a card reader, software, data entry 
and scoring cards. SAGE results can be used directly as input into the JOBS 
job matching system through the use of "Scor-O-Matic." The entire system 
requires: IBM or IBM compatible computers with the 2 disk drives, 640 K 
RAM, communication port, monitor, and 120 CPS dot matrix printer. 

4. Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is used. SAGE can be 
considered as the first step in vocational evaluation; this system can also be 
used alone for vocational assessment. 

b. Sequence of Work Sample Administration - The five components can be given 
in any order. Within the VAB, the aptitude tests can be given in any order 
or on different days. 



See "temperaments" in glossary for a list of the ten temperaments. 

121 

121 




c. Client Involvement - There is little client involvement during testing. Client 
involvement before and after testing centers around a good orientation on 
why they arc being tested, what it will do for them and how it relates to 
training or placement. 

d. Evaluation P-ttting - The papcr-and-pencil and apparatus Usts will remi^^i the 
client that he/she is in a formalized testing situation. 

e. Time to Complete Entire System - Several persons can complete the entire 
system in about four hours. The system provides for multiple administratiokj; 
in the VAB there is enough equipment/material to assess 23 people simul- 
taneously. 

5. Administration 

a. Procedures - One comprehensive manua' provides detailed instructions on the 
purpose, general characteristics, directions and scoring for each unit of the 
SAGE. 

b. Method of Instruction Giving - Each instrument can either be self-admin- 
istered (if the client reads at the fourth grade level) or by having the in- 
structions read aloud. The VII uses a titled filmstrip and cassette tape that 
contains instructions as well as the test items. 

c. Separation of Learning/Performance - The C-CAT and the VAB tests have the 
following procedures: initial instructions, timed practice exercises and the 
actual test. Thus, there is a clear separation of learning from performance. 
SAGE contains a higher proportion of practice items to the test items than 
do most other tests. 

d. Providing Assistance to the Client - The evaluator may read instructions and 
some test items. Evaluators and teachers are instructed to answer all ques- 
tions before and after the practice session. No assistance is given during the 
timed portions of the tests. 

e. Repeating Work Samples - Readministration is permitted if the evaluator 
believes results are not valid. According to the developer, SAGE may be used 
for pre and post testing. 

6. Scoring and Norms 

a. Timing - The C-CAT and VAB use an electric timer, controlled either by the 
client or evaluator. A buzzer and li'ht mark the completion of each timed 
period. A multi-choice electronic score recorder and timer are also available. 
The Eye-Hand-Foot and Motor Coordination tests \\a\c buIU in timers. 

b. Timing Interval - For each test there is a specific set time for both the 
practice items and the actual test items. 

c. Time Norms - For the VAB and C-CAT the number of correct responses is 
converted to a scale score of one to five or six. These snores correspond to 
the Aptitude and GED levels used in the Labor Department system. The AWA 
i3 scored using a Likert-type scale from one to four. Item values are totaled 
and the resulting score compared to three scoring ranges: job ready, and 
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doubtful. In the VII the number of items chosen in each of the 12 interest 
areas are totaled and compared to preselected arbitrary cut-off points. 

In the Temperament test the number of true responses chosen for each ten 
factors are rank ordered. A total of five or more true answers per factor 
indicates a clear distinction, three to four true responses is a mild indicator 
and one to two indicate a strong reaction against the factor. 

d. Error Scoring - This is not relevant; no errors are recorded. 

e. Scoring Aids - Scoring keys are used with all paper-and-pencil tests. Several 
units have built in self-timing and scoring devices. The Scor-O-Matics pro- 
vide for .?elf-scoring and timing for all multiple choice tests except the C- 
CAT. 

f. Quality Norms - Not relevant; no errors are recorded. 

g. Emphasis in Scoring - Not relevant because only one type of score, a correct 
response, is recorded. 

7. Observation of Clients 

a. Work Performance - No work performance factors are used in the SAGE 
system. 

b. Work Behavior - The system uses one form, the "Observation Record," to 
record 19 separate behaviors; these are grouped into six "factors": social/co- 
operation, communication, personal, working/learning, physical and handicaps. 

c. Recording System - The Observation Record uses a four point rating scale 
(unsatisfactory, marginal, average, and superior). The points for each scale 
are totaled. 

d. Frequency of Observation - No procedures are given in the manual. 

8. Reporting 

a. Forms - SAGE uses a variety of forms: answer sheets, scoring keys, conver- 
sion tables and a SAGE Profile Form. All forms are well-designed and inter- 
nally consistent with each other. 

b. Final Report Format - The SAGE Profile Form consists of an Observation 
Record, Raw Score Conversion Profile Occupational Match Form and a first 
page containing general information about the client and recommendations for 
additional evaluation, training program placement and/or job placement. The 
final report is a product of the individual worker traits (i.e.. Aptitudes, GED, 
Temperaments and Intcicsts, plus Attitudes) measured by SAGE and a match 
of the job demands for individual occupations. 

9. Utility 

a. Vocational Exploration - SAGE provides minimal vocational experiences; there 
are few hands-on materials and little vocational explanation given prior to 
testing, other than the general orientation. 
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b. Vocational Recommendations - The SAGE Profile Form contains specific job 
recommendations by DOT title and code. SAGE'S data base contains aptitudes 
and GED results for 487 commonty held occupations. An additional manual 
provides information on matching ever 12,099 job titles. The system can also 
be used with other commercial assessment systems (there are cross-references 
relating the Aptitudes as measurr.d by SAGE to JEVS, Micro-TOWER, Project 
Discovery, SAVE, Valpar, Singev, VIEWS, and VITAS), with job site training, 
f'lr^her testing, etc. 

c. Counselor Utilization - Counselor utilization would depend upon two variables: 
(1) How SAGE is integrated with other vocational evaluation tools and (2) 
how well the counselor knows and uses the DOT and related data. A soft- 
ware package JOBS is available for on-site job matching (See Botterbusch, 
1986 for a description of JOBS.) 

10. Training in the System 

a. Training Required - Although not required, training is recommended. 

b. Training Available - Training is available either on-site or in New York. 

c. Duration - Training is for one day. 

d. Follow-up - The developer provides a toll free number to answer any ques- 



1 1. Technical Considerations 

a. Norm Base - There are four norm groups for the SAGE: (1) low functioning 
with a sample size of 240, (2) successfully employed with a sample size of 
400, (3) vocational-technical students with a sample size of 650 and (4) gen- 
eral working population with a sample size of 1800. Results are presented as 
standard scores, z scores, percentiles and Aptitude, and GED level scales. 
The manual contains no information as to the source and characteristics of 
this sample. 

b. Reliability - Test-retest coefficients and standard errors of measurement were 
used to determine reliability for most of the tests. Most of the KR-20 relia- 
bility coefficients are reasonably high. 

c. Validity - A variety of item analysis data and validity data are presented for 
each section. Validity data includes correlations with ratings and other tests; 
the majority of these are at acceptable levels. 

12. Reviewer's Summary and Comments 

SAGE is not a traditional work sample system, but rather is a combination of 
paper-and-pencil tests and isolated trait work samples that can be administered in 
about four hours. It can be used either by itself or as a screening device given 
prior to a longer period of vocational evaluation. The developer has gone as far 
as providing charts of the relationship between SAGE and the more popular 
commercial work sample systems. Because it is basically an ability assessment 
device, SAGE should be supplemented with appropriate occupational exploration. 
Although some behaviors are observed and recorded, most evaluators will want to 



tions. 
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collect additional behavioral data. Overall, SAGE gives the initial appearance of a 
well-planned assessment xool that is aimed mostly at secondary school studentb, 
especially the disadvantaged. 

The major advantages of the system are its relationship to DOT related variables, 
its relatively short administration time, and its apparent flexibility when used in 
combination with other systems. Finally, while SAGE is not specifically designed 
for a handicapped population, it can be used with most types of disabilities by 
making a few commonsense modifications. 

13. Address 

Progressive Evaluation Systems Corp. 
21 Paulding Street 
Pleasantville, New York 10570 

14. Cost 

The cost per component is: 



Vocational Aptitude Battery $5,495.00 

Vocational Interest Inventory $500.00 

Cognitive and Conceptual Abilities Test $500.00 

Assessment of Woik Attitudes $500.00 

Temperament Factor Assessment Unit $ 375.00 

Total $7,370.00 

Other costs are: 

On-site training (travel expenses not included): 

First day $500.00 

Second consecutive day $250.00 

Additional SCOR-O-MATICS. $275.00 

Additional Electronic Timera $49.50 

SAGE carrying cases $650.00 

Job Opportunity Based Search (JOBS) $485.00 to $2,000.00 



15. References 

Botterbusch, K. F. (1982). SAGE • System of assessment and group evaluation. 
Vocational Evaluation and Work Adjustment Bulletin. 15(1), 32-34. 
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Talent Assessment Program 



(TAP) 



Development 

z. Sponsor - TAP was initially developed by Mr. Wilton E. Nighswonger. It is 
marketed by Mr. Ben Borden of Talent Assessment Inc. of Jacksonville, Flor- 
ida. 

b. Target Group - TAP is intended for use with a wide range of populations and 
mental ability levels above trainable mentally retarded. It has been used with 
disadvantaged, handicapped, and "regular" high school students, vocational- 
technical school students and adults. 

c. Basis of the System - TAP was designed to measure "functional aptitudes 
applicable to things and materials in the world of work." The systefi. mea- 
sures a number of coordination, dexterity and perceptual aptitudes, as defined 
in the manual. Because the manual contains no development history of the 
system, it is difficult to assess the nature of these aptitudes. 

Organization 

a. Name and Number of Work Samples - Ten tests are included in the syplem: 

(1) Structural and Mechanical Visualization - Assembles small metal frame 
following a model. 

(2) Discrimination by Size and Shape - Sorts 12 different sizes and shapes 
of screws. 

(3) Discrimination by Color - Sorts six different colored marbles. 

(4) Discrimination by Touch - Sorts 12 sandpaper disks by degree of rough- 
ness. 

(5) Dexterity without Tools-Small - Finger assembly of electrical slip bolts, 
clips and nuts. 

(6) Dexterity without Tools-Large - Hand assembly of 12 large electrical 
conduit couplers. 

(7) Dexterity with Small Tools - Inserts large staples with tweezers and 
screws a metal bar over them, 

(8) Dexterity with Large Tools - Uses a crescent and other wrenches to 
assemble bolts to a frame. 

(9) Visualization of Flow Patterns - Determines the flow of electricity in 
ten simple diagrams. 

(10) Retention of Mechanical and Structural Detail - Assembles the frame in 
the first work sample without a model. 

b. Grouping of Work Samples - Each test is administered and scored indepen- 
dently. With the exception of Tests I and 10, the tests can be given in any 
order. 

c. Manual - The system manual contains general directions, scoring information, 
norms tables as well as several examples of profiles. Specific instructions are 
given for materials, set-up and administration procedures for both client and 




evaluator. Photographs are used to insure proper layout. The manual is very 
well organized. 

The TAP computer manual contains instructions for machine scoring and 
searching for specific jobs based on the client profile. This manual is well- 
written and easy to follow. In addition to the computer instructions, the 
manual defines the DOL job analysis terms used in the software. 

Physical Aspects 

a. Packaging of the Work Sample - Each work sample is packaged independently; 
most of the work samples are contained in heavy plastic cases. Optional 
carrying cases are available; this increases the portability of the system. 

b. Durability - TAP uses sturdy plastic cases and many of the metal components 
are made of case hardened steel. TAP tools and equipment are extremely 
durable. 

c. Expendable Supplies - Aside from recording forms and paper for the printer, 
TAP uses no expendable supplies. 

d. Replacement - If any replacement parts are needed, they could be ordered 
from the developer or purchased locally. The profile form and time sheets 
can be locally reproduced. 

e. Computer Requirements - The scoring and job matching components are 
available for Apple II, He and lie, TRS-80, and IBM-PC. Each system re- 
quires the following: computer with a minimum of 64K RAM, monitor, one 
disk drive, and printer. The evaluator must supply his/her DOS. The scoring 
and job matching programs are contained on four floppy disks. 

Vocational Evaluation Process 

a. Preliminary Screening - There is no mention of preliminary screening in the 
manual. 

b. Sequence of Work Sample Administration - The Structural and Mechanical 
Visualization Work Sample (number 1) must be given first and work sample 
number 10 (Retention of Structural and Mechanical Detail) last. The rest may 
be given in any order. This is because the last work sample requires the 
client to construct the same structure as does the first work sample, except 
it is done without a model. Thus, this separation is needed as a measure of 
retention. 

c. Client Involvement - The type and degree of client involvement and feedback 
during administration is left to the discretion of the evaluator. The manual 
contains no specific procedures to be followed to ensure client participation. 
However, "Assessment results should be shared with [the] client as soon as 
possible. The original copy of the profile is usually given to those assessed." 

d. Evaluation Setting - Although the evaluation setting is not specified, TAP 
lends itself to a formal testing atmosphere. 
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e. Time to Complete the Entire Battery - The tests can be administered in from 
two to two and a half hours. Computerized scoring should take about ten 
minutes. 

Administration 

a. Procedures - The materials, tools, layout, client instructions, and demonstra- 
tions are specified in detail; photographs are used to obtain precise equipment 
set-up. The manual cautions against testing persons who are under medica- 
tion, ill, depressed, etc. 

b. Method of Instruction Giving - While the basic method of instruction giving 
is oral with demonstration, the teacher or evaluator is to "make certain that 
clients have complete understanding of directions by giving them orally and, 
if necessary, by: demonstration, having clients demonstrate, having clients 
repeat directions, permitting clients to practice." No reading is required for 
any test. Any spoken language may be used; sign language can also be used. 
In short, with the TAP system any form of instruction and communication is 
acceptable as long as the client understands the instructions. 

c. Separation of Learning/Performance - There is a client practice period prior 
to beginning timing. After this practice period timing begins. Because the 
TAP manual emphasizes having the client fully understand the instructions, 
there is considerable separation of learning and performance. 

d. Providing Assistance to the Client - TAP is one of the few work sample 
systems where assistance can oe given after the practice section is completed: 
"If a client is observed doing an assessment in the wrong manner, stop the 
work and start again, after complete understanding is achieved." 

e. Repeating Work Samples - To quote the manual: "Assessment should be 
redone if individuals express the feeling that they can do better. This will 
happen with a few who ultimately realize that they did not really try." 

Scoring and Norms 

a. Timing - The evaluator times the client using an electronic timer that records 
minutes and tenths of minutes. 

b. Timing Interval - Timing begins when the client fully understands ihe instruc- 
tions and stops when the task is completed. 

c. Time Norms - The actual completion time to the nearest tenth of a minute is 
recorded. After any "penalty" (see directly below) scores are added to the 
completion time, the total raw time score is compared to percentile norms. 
With the computer scored option, the completion time minus any penalty is 
entered into the computer for each work sample. In both the manual and 
computer systems, the evaluator selects the appropriate norm group. 

d. Error Scoring - Tests 5, 6, 7 and 8 are completion tests. The evaluator 
checks the task and, if not complete, the client is told to complete the task. 
The additional time needed is recorded and added to the original time. Tests 
1, 2, 3, 4, 9 and 10 incorporate errors into the time score; a penalty time is 



129 



128 



assigned based on the number of errors. This time is added to the raw score 
and the new score is compared to percentile norms. 

e. Scoring Aids - The design of the tests prevents the use of scoring aids. 

f. Quality Norms - There are no separate quality norms. In those tests which 
are scored for errors, the number of errors are multiplied by a constant 
number and the resulting "penalty" is added to the raw time score. 

g. Emphasis in Scoring - The emphasis is clearly on time scores. 
Observation of Clients 

a. Work Performance - A few work performance factors are mentioned, but none 
are defined; no information is given for their observation. 

b. Work Behaviors - A few work behaviors are mentioned, but none are defined; 
no information is given for their observation. 

c. Recording System - No method for rating behaviors is used. 

d. Frequency of Observation - This is not specified. Because TAP is intended to 
be used primarily as an objective assessment battery, the system's developer 
chose not to emphasize client observation. 

Reporting 

a. Forms - A raw score form is used to record starting and stopping times, 
errors, etc. for each work sample. Both manual and machine scoring are 
based on this form. 

b. Final Report Format - The manual report is a profile sheet containing per- 
centile scores for each work sample; a placement and training number (PAT) 
is assigned to each individual profile sheet. This number is assigned to each 
individual sheet based on performance and specific occupations in the DOT, 
worker trait groups in the GOE. 

The computer generated report begins with demographical data. Next the test 
times and percentile scores are presented in both tabular and graphic form. 
The individual aptitude strengths are then listed. A short narrative for each 
test explains the relation between the test and occupational areas. Finally, a 
list of high PAT profiles is printed. At this point the evaluator inserts one 
of the three job data base disks and selects DOT titles based on identical 
PAT numbers. 

Utility 

a. Vocational Exploration - Because the system is really a group of standardized 
perceptual and dexterity tests, they are too abstract to provide vocational 
information to the client without interpretation by the teacher or evaluator. 

b. Vocational Recommendations - Using the PAT number(s) for each individual 
profile, the manual lists exact job titles with DOT codes. This job listing is 
fairly comprehensive. 
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c. Counselor Utilization - The profile sheet or the computerized final report 
with its occupational recommendations is designed for the counselor, teacher, 
employer or client The user has specific information on individual jobs that 
can be used in a variety of ways. 
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10. Training in the System 

a. Training Required - Training is required prior to use. 

b. Training Available - Training is held at the purchaser's site. All expenses for 
training are included in the purchase price. 

c. Duration - Training takers about a day and a half. 

d. Follow-up - Information and consultation can be provided as needed. 

11. Technical Considerations 

a. Norm Base - The manual contains the following norm groups: (1) 12th grade 
male/female combined samples of over 5,000; (2) 12th grade females; (3) 12th 
grade males; (4) grade 7 and 8 male/female combined samples of about 5,000; 
(5) grade 7 and 8 males; (6) grade 7 and 8 males and (7) approximately 1,000 
mentally retarded males and females in special education programs. The 
computer scoring software lists three norm groups: (1) high school seniors; 
(2> junior high school population, and (3) mentally retarded populations. 
Although these groups are of adequate size, the manual contains no details of 
sample selection nor of sample characteristics. The reviewer considers this to 
be a serious problem. 

There are three separate PAT-DOT data bases. The first contains about 228 
jobs; these are largely skilled, technical and professional jobs in "Things" 
areas. The second data base is designed for "handicapped." These are mostly 
semi-skilled and unskilled mechanical or industiial jobs. These two data bases 
are contained both in the manual and on the scoring software. The final 
data base is flexible software designed to allow the evaluator to develop 
his/her own local data base. 

b. Reliability - The developers report a coefficienr of stability of over .85 in 
preliminary test-retest studies over a six month period. Unfortunately, not 
enough information on the methodology and subjects is included to judge 
these results. 

c. Validity - No data are available. 

12. Reviewer's Summary and Comments. 

As opposed to other work evaluation systems that attempt to present a complete 
picture of the client, TAP crm be characterized as a battery of perceptual and 
dexterity tests designed to measure gross and fine finger and manual dexterity, 
visual and tactile discrimination, and retention of details. Thus, the system is 
limited to the assessment of th^se fairly specific factors. The developer does not 
claim that this system will assess all vocational significant capacities and be- 
haviors. In. fact, the manual states that other assessment devices should be used 
in addition to TAP to obtain a complete evaluation of the client. 
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TAP'S major advantages include durable equipment, a well-written manual, flexi- 
bility in client instructions and a computerized scoring program. 



The system's problems are largely technical. Norm groups should be defined in 
greater detail and additional reliability studies are needed. Although the printout 
lists numerous "aptitudes," these are not defined in the manual, nor are they 
related to specific TAP tests. There is no reported research to support either the 
definitions or their relationship to the ten TAP tests. The most important part 
of the system is the transformation of the test profile to the three digit PAT 
codes. Each of these codes is then related to specific jobs. Yet the manual 
contains no description of the research or other processes used to establish this 
relationship. In addition, no empirical validity information is available. This lack 
of both construct and empirical validity information casts doubts on the TAP 
system. 

13. Address 

Tulcnt Assessment Inc. 
P.O. Box 5087 

Jacksonville, Florida 32247-5087 

14. Cost 

Complete Talent Assessment System (includes all TAP testing components, com- 
puter software for scoring, portable carrying cases, 1 and 1/2 days of on-site 
staff training, and shipping and handling) $5,360.00 

TAP components (includes staff training and shipping) $4,670.00 

Computer software (for Apple, TRS-80 and IBM) $495.00 

Portable Carrying Cases $195.00 

15. References 

Morley, R. (1973). (Ed). Vocational assessment systems. Des Moines, I A: State of 
Iowa, Department of Public Instruction. 

Zikmund, D. and Reinder, L. (1974). Talent Assessment Program Test Battery [sic]. 
In A. Sax (Ed.), Innovations in vocational evaluation and work adjustment. 
Vocational Evaluation and Work Adjustment Bulletin. 7(4). 
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The TOWER System 

(TOWER) 



1. Development 

a. Sponsor - TOWER^^ was originally developed with funding from the U.S. 
Department of Health Education and Welfare, Vocational Rehabilitation Ad- 
ministration. Subsequent refinements have been made by the ICD, Interna- 
tional Center for the Disabled. 

b. Target Group - Apparently the system was first developed for physically 
disabled persons; it is now used for all types of disabled persons, such as 
emotionally disabled. 

c. Basis of the System - TOWER is based on job analysis of positions that were 
considered open to handicapped persons in the New York City area. 

2. Organization 

a. Name and Number of Work Samples - The system contains 93 work samples 
arranged into 14 job training areas: 

(1) Clerical - Business Arithmetic; Filing, Typing, One-hand Typing; Payroll 
Computation; Use of Sales Book; Record Keeping; and Correct Use of 
English. 

(2) Drafting - T-Square and Triangle; Compass; Working Drawing; Drawing 
to Scale; and Geometric Shapes. 

(3) Drawing - Perspective; Forms, Shapes and Objects; Shading; Tone and 
Texture; Color; and Free Hand Sketching. 

(4) Electronics Assembly - Color perception and Sorting; Running a 10 Wire 
Cable; Inspecting a 10 Wire Cable; Lacing a Cable; and Soldering Wires. 

(5) Jewelry Manufacturing - Use of Saw; Use of Needle Files; Electric Drill 
Press; Piercing and Filing Metals; Use of Pliers; Use of Torch in Sol- 
dering; and Making Earring and Broach Pin. 

(6) Lcathergoods - Use of Ruler; Use of Knife; Use of Dividers; Use of 
Paste and Brush; Use of Scissors and Bond Folder in Pasting; Con- 
structing Picture Frame; and Production Task. 

(7) Machine Shop - Reading and Transcribing Measurements; Blueprint 
Reading; Measuring with a Rule; Drawing to Measurement Metal Layout 
and Use of Basic Tools; Drill Press Operation; Fractions and Decimals; 
Measuring with the Micrometer Caliper; and Mechanical Understanding. 

(8) Lettering - Lettering Aptitude; Alphabet and Use of T-Square; Use of 
Pen and Lik; Use of Lettering Brush; and Brush Lettering. 

(9) Mail Cleik - Opening Mail; Date-Stamping Mail; Sorting Mail; Delivering 
Mail; Collecting Mail; Folding and Inserting; Sealing Mail; Mail Clas- 
sification, Use of Scale; and Postage Calculation. 



12 "TOWER" is an acronym for "Testing, Orientation and Work Evaluation In 
Rehabilitation." 
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(10) Optical Mechanics - Use of Metric Ruler; Use of Calipers; Lens Recog- 
nition; Lens Centering and Marking; Use of Lens Protractor; and Hand 
Beveling and Edging. 

(11) Pantograph Engraving - Introduction to the Engravograph; Setting-up, 
Centering Copy and Determining Specified Ratios; Use of Work-holder 
and Adjustment of Cutter; and Setting-up and Running Off a Simple 
Job. 

(12) Sewing Machine Operating - Sewing Machine Control; Use of Knee Lift 
and Needle Pivoting; Tacking and Sewing Curved Lines; Upper Thread- 
ing; Winding and Inserting Bobbin; Sewing and Cutting; and Top Stitch- 
ing. 

(13) Welding - Measuring; Making a Working Drawing; Identifying Welding 
Rods; Use of Acetylene Torch; Use of Rods and Electrodes; Use of 
Torch and Rod; Measuring and Cutting Metal; and Soldering. 

(14) Workshop Assembly - Counting; Number and Color Collation; Folding 
and Banding; Weighing and Sorting; Counting and Packing; Washer 
Assembly; Inserting, Lacing and Typing; and Art Paper Banding. 

b. Grouping of Work Samples - The work samples are grouped into 14 major 
areas of training. While each of the 14 areas is independent, the work 
samples within each area arc arranged in order of complexity. In most 
instances, simpler tasks must be completed before beginning complex ones. 

c. Manual - The printed manual is bound in a loose-leaf folder. There is a 
separate section for each of the 14 areas. Each contains the following major 
headings: orientation, preparation, instructions for each work sample scoring 
criteria, and any scoring aids. Some details on the set-up are not included; 
also it is not always clear if instructions should be read to the client or read 
by the client. 

Physical Aspects 



a. 


Packaging of the Work Samples 












Because ICD does not sell hard- 
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Durability 




ware or equipment, each facility 
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Expendable Supplies 
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pend upon the individual facility. 


d. 


Replacement 




e. 


Computer Requirements 







Work Evaluation Process 



a. Preliminary Screening - This is emphasized for planning purposes, but the 
specific information needed prior to administration of the system is not 
specified. 

b. Sequence of Work Sample Administration - Administration is progressive 
within the major areas; the choice of areas depends upon client interest 
and/or the evaluation plan. 

c. Client Involvement - No client involvement procedures are specified in the 
manual. 
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d. Evaluation Setting - A realistic work atmosphere and setting are stressed. 

e. Time to Complete the Entire System - The average client completes the entire 
system in three weeks; however, clients seldom take all work samples in the 
system. 

5. Administration 

a. Procedures - The purpose and procedures arc clearly described. All required 
tools and materials are listed. Almost no layout details are given. 

b. Method of Instruction Giving - The system uses mainly written instructions 
that are supplemented by evaluator explanation and demonstration when 
needed. 

c. Separation of Learning/Performance - Many of the work sample instructions 
contain no separation between a formal practice period and an established 
timing period. This manual is not at all clear on this point. 

d. Providing Assistance to the Client - The evaluator is encouraged to ensure 
that the client knows how to perform the task before he/she begins to work; 
procedures for assisting the client after he/she has started the task are not 
specified. 

e. Repeating Work Samples - The readministration of work samples is encouraged 
for the purpose of upgrading client performance. 

6. Scoring and Norms 

a. Timing - The evaluator times the client, but no procedure for timing is 
established. 

b. Timing Interval - Timing begins following instructions and stops upon comple- 
tion of the task. Often, however, this point is difficult to locate. 

c. Time Norms - Time results are rated on a five-point scale, based upon the 
number of minutes to completion. 

d. Error Scoring - All items arc checked against carefully defined scoring cri- 
teria. 

c. Scoring Aids - Extensive use is made of transparent overlays and other 
scoring aids. 

f. Quality Norms - A work samples arc rated on a five-point scale, based upon 
the number of errors. 

g. Emphasis in Scoring - Time and the quality of the finished product are given 
equal weight. 

7. Observation of Clients 

a. Work Performance - The only work performance factor specifically listed is 
"dexterity." 
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b. Work Behaviors - A comprehensive checklist of work behaviors (e.g., neatness, 
attendance) are contained in the vocational evaluation report. 



c. Recording System - A five-point system is used to rate ^'work and personal 
characteristics"; the points on the scale are not clearly defined. 

d. Frequency of Observation - Frequent observations are not emphasized, but are 
taken for granted. There is no established procedure for behavior observa- 
tion. 

8. Reporting 

a. Forms - Standardized forms arc used for attendance and punctuality; for a 
summary of time and quality results for each work sample; and for a "voca- 
tional evaluation report." 

b. Final Report Format - The three page final report contains ratings of "Work 
and Personal Characteristics," ratings for each of the 14 job areas and a 
narrative report. 

9. Utility 

a. Vocational Exploration - The client is exposed to many different training 
areas which are representative of a variety of jobs. The manual contains 
some specific occupational information that is given during the administration 
of the work samples. 

b. Vocational Recommendations - Vocational recommendations are limited to jobs 
that arc directly related to the work samples. The recommendations are not 
highly related to the DOT and are primarily training oriented. 

c. Counselor Utilization - Counselor involvement in the evaluation process is 
recommended; the final report is aimed at the referring counselor and client. 

10. Training in the System 

a. Training Required - Training is required, for inexperienced vocational evalua- 
tors. 

b. Training Available - Includes training in other work sample systems as well as 
work sample development. 

c. Duration - One and two week introductory courses. 

d. Follow-up - No 

11. Technical Considerations 

a. Norm Base - The system was normed on clients at the ICD, International 
Center for the Disabled: sample sizes or characteristics are not given. 
Industrial norms arc not available. 

b. Reliability - No data available. 
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c. Validity - A. seven city research study produced equivocal results. The em- 
pirical validity of the TOWER is still open to much question. 



1 2. Rcviewcr^s Summary and Comments 

The TOWER System is the oldest complete work evaluation system and over the 
years has served as a model for the development of many work samples. The 
TOWER uses a realistic job setting to thoroughly evaluate clients for a rather 
narrow group of jobs. The facts that the TOWER was based on job analysis and 
that the system has been used for many years to place and train handicapped 
people arc indications that the system is very useful in evaluating clients for a 
small group of jobs. The lack of precise definitions for work performance factors 
and client behaviors and the lack of adequate norms arc the major weaknesses of 
the system. The high use of written instructions and the high level of the areas 
evaluated restricts its use with low literate and mentally retarded clients. 

13. Address 

International Center for the Disabled 

340 East 24th Street 

New York, New York 10010 

14. Cost 

Manual - (Includes work sample directions, response sheets and scoting criteria). 



Without Training $250.00 

Upon Completion of Training $100.00 

Individual work samples, plus 3 extra 

sets of evaluations. $25.00 to $75.00 

Training Tuition 

One Week Course $250.00 

Two Week Course $450.00 



Note: No hardv/arc is sold by ICD; each facility constructs its own work samples. 
ICD estimates cost to set up a complete unit is $5,000.00. 
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Valpar Component Work Sample Series 

(V&Ipar) 



1. Development 

a. Sponsor - The work samples were developed or modified by Valpar Interna- 
tional. 

b. Target Group - Originally iniended for use with the general population, the 
Valpar work samples have been used extensively with industrially injured 
workers. The manuals do not contain a statement that the work samples are 
designed to serve any specific population; it is assumed that Valpar work 
samples are useful with a wide variety of client groups. Modifications are 
available for Valpar 1, 2, 3, 4, 7, 8, 9, and 10 for visually disabled persons. 
Videotapes and signed administration instructions are available for deaf per- 
sons on Valpar 1 to 16, except 14. 

c. Basis of th^ System - According to the developers, the work samples are 
based on the trait-and-factor approach taken from job analysis. The manual 
for each work sample relates that work sample to several Worker Trait 
Groups Arrangements as well as specific occupations. 

2. Organization 

a. Name and Number of Work Samples - At present 19 work samples are con- 
tained in the series: 

(1) Small Tools (Mechanical) - Use of small tools, such as screwdrivers, 
pliers, and wrenches. 

(2) Size Discrimination - Attaches nuts of different sizes to randomly 
arranged bolts. 

(3) Numerical Sorting - Sorts by numbers and number series. 

(4) Upper Extremity Range of Motion - Range of motion of upper torse 
through placing nuts on bolts from many different angles. 

(5) Clerical Comprehension and Aptitude - Evaluates general clerical, book- 
keeping and typing. 

(6) Independent Problem Solving - Measures ability to perform tasks re- 
quiring the visual, comparison and selection of abstract designs. 

(7) Multi-Level Sorting - Sorts by a combination of numbers, letters and 
colors. 

(8) Simulated Assembly - Ability to perform a repetitive task requiring 
finger and manual dexterity. 

(9) Whole Body Range of Motion - Agility of gross body movements for 
trunk, arms, hands, fingers and legs. 

(10) Tri-Level Measurement - A.bility to perform from very simple to very 
precise inspection and measurement. 

(11) Eye-Hand-Foot Coordination - Ability to use eyes, hands and feet 
simultaneously and in a coordinated manner. 



The numbers of the work samples correspond to the number sequence given in 
"2.a. Name and Number of Work Samples." 
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(12) Soldering and Inspection-Electronic - Soldering tasks of varying dif- 
ficulty. 

(13) Money Handling - Monetary concepts from recognition to consumer 
economics. 

(14) Integrated Peer Performance - A group assembly and inspection task. 

(15) Electrical Circuitry and Print Reading - Understands principles of 
electrical circuits and use of diagrams, etc. 

(16) Drafting - Potential for drafting and blueprint reading. 

(17) Pre-Vocational Readiness Battery - This is described as a separate work 
sample system on page 107. 

(18) Conceptual Understanding Through Blind Evaluation (CUBE) - Assesses 
coping skills in meeting basic needs of mobility, judgment, orientation 
and balance. 

(19) Dynamic Physical Capacities - Measures physical demands as defined by 
the U.S. Department of Labor publications. 

b. Grouping of Work Samples - The work samples were developed and are in- 
tended for use as individual components. They are not organized into a 
single, comprehensive evaluation system. 

c. Manual - A separate manual is used for each work sample. Each contains 
sections on purpose, job classifications, work sample description, general 
administration and scoring, client instructions, rating directions, and norma- 
tive data. A separate Evaluator^s Manual contains sections on scoring, norms 
for each work sample as well as descriptions of the norm groups and methods. 
Most of the manual contents are detailed and easy to follow. 

Physical Aspects 

a. Packaging of the Work Samples - All work samples are packaged separately 
and are self-contained. Where appropriate, work samples are contained in 
lockable cases. 

b. Durability - Components are well-constructed and durable, requiring little or 
no maintenance. One exception is the Money Handling work sample where 
problems with the dial can occur. 

c. Expendable Supplies - Most of the Valpar work samples require no expendable 
supplies. The few that do use expendable supplies use mainly paper forms. 

d. Replacement - All replacement parts can be ordered from Valpar. Forms may 
be reproduced locally or ordered from Valpar. 

e. Computer Requirements - This section in not applicable for the Valpar series. 
Work Evaluation Prpg ^g?; 

a. Preliminary Screening - The work samples do not require preliminary screen- 
ing. 

b. Sequence of Work Sample Administration - The order and the number of work 
samples to be given is left to the discretion of the evaluator. It must be 
remembered that Valpar is a group of independent work samples and not a 
system. 
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c. Client Involvement - Because work sample administration resembles a formal 
testing situation, client involvement in minimal; feedback on performance is 
left up to the discretion of the facility and individual evaluator. 

d. Evaluation Setting - The work samples can be used either in a classroom or 
workshop setting. 

e. Time to complete Entire System - It is estimated by the reviewer that most 
work samples can be completed in one hour or less. The Drafting, Integrated 
Peer Performance and Clerical Comprehension and Aptitude often take over 
one hour. Because the Pre-vocational Readiness Battery and CUBE have 
several sections, they often take four or more hours. 

Administration 

a. Procedures - The materials required, evaluator instructions, and the layout are 
clearly described *n the work sample manuals; detailed illustrations of the 
work sample are sed to insure accuracy. All work samples recycle them- 
selves so that they are ready for the next administration. Thus, little evalu- 
ation time is spent in disassembling completed tasks. 

b. Method of Instruction Giving - A combination of oral instructions with ac- 
companying demonstrations is used by the evaluator to administer most work 
samples. Although the manuals imply that instructions are to be read ver- 
batim, other Valpar literature and training indicates that they do not have to 
be re-vd verbatim. In the Clerical Comprehension and Aptitude, Money Hand- 
ling, and Drafting Work Samples, the client is required to read instructional 
and testing materials that simulate the tasks required in these three work 
samples. 

c. Separation of Learning/Performance - Eleven of the Valpar work samples have 
a formal practice period during which the client must reach an established 
criteria. 

d. Providing Assistance to the Client - The evaluator is encouraged to insure 
that the client lias a thorough understanding of the task and demonstrate 
each task if, necessary, on those work samples without formal practice ses- 
sions before beginning timing. The manuals do not specify what (if any) 
assistance may be given to the client after timing has started. 

e. Repeating Work Samples - If desired by the evaluator, readministration is 
encouraged. 

Scoring and Norms 

a. Timing - The evaluator times the client. On some work samples (e.g., Cleri- 
cal Comprehensive and Aptitude), where there are several distinct tasks, each 
task is timed separately. The disassembly of many work samples is also 
timed. The manuals are specific as to when timug should begin and end. 

b. Timing Interval - Timing begins after the instructions have been given and 
ends when the ask is completed. The typing test in Clerical Comprehension 
and Aptitude, Simulated Assembly and the endurance exercise in Dynamic 
Physical Capacit.es are exceptions. 
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c. Time Norms - The completion time in seconds is recorded for each section of 
all work samples. The total time is converted into percentiles at 5% inter- 
vals; MTM standards also use percents as a conversion method. 

d. Error Scoring - Errors are well defined; the number of errors is recorded for 
each part of the sample and totaled. Total errors are converted to a percen- 
tile score. Valpar also uses a performance percentile score that is a combi- 
nation of time and error scores. When appropriate, there are MTM error 
norms for all work samples. 

e. Scoring Aids - Use is made of scoring aids; some work samples have auto- 
matic scoring devices. 

f. Quality norms - Separate quality norms are used; errors are converted to 
percentiles in 5% intervals. 

g. Emphasis in Scoring - Present training by Valpar emphasizes the use of MTM 
standards over percentile scores. 

Observation of Clients 

a. Work Performance - No work factors are specified for individual work sam- 
ples. 

b. Work Behaviors - The same 17 worker characteristics (e.g., ability to work 
alone; ability to respond to change; ability to communicate; ability to make 
decisions) are defined in each work sample manual. There are no behaviors 
that are to be observed for each separate work sample. Most of these char- 
acteristics are not clearly defined and all require subjectivity on the part of 
the evaluator. Evaluators are instructed to rate only those characteristics 
"which are applicable to the client." 

c. Recording System - The evaluator uses a five-point scale to rate clients on 
each of the 17 worker characteristics. 

d. Frequency of Observation - Frequency of observation is not specified. How- 
ever, frequent evaluator contact is required on many work samples due to the 
administration and scoring procedure. 

Reporting 

a. Forms - A separate, standard form is included with each work sample for 
recording scoring information and rating worker characteristics. Body posi- 
tion charts are included with Upper Body Range of Motion, Whole Body Range 
of Motion and Dynamic Physical Capacities for recording pain and fatigue. 

b. Final Report Format - Because the work samples are not part of a unified 
system, no information or recommendations are given for reporting results in 
a unified manner. 

Utility 

a. Vocational Exploration - There is limited opportunity for vocational explora- 
tion due to the abstract nature of <:ome of the work samples. 
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b. Vocational Recommendations - Because these are individual components and 
not an evaluation system, vocational recommendations cannot be made on the 
basis of one work sample. The use of the Valpar work samples for making 
vocational recommendations depends largely on their use in the individual 
evaluation unit. 

c. Counselor Utilization - Because the system uses the purchasing facility's or 
schoors report format, counselor utilization cannot be specified. 

10. Training in the System 

a. Training Required - Training is not required as a condition of purchase. 

b. Training Available - Training is available from Valpar International. 

c. Duration - Duration depends upon the needs of the evaluator. 

d. Follow-up - Follow-up after training is available on a consultation basis. 

11. Technical Considerations 

a. Norms - Different work samples were normed on different samples; the norm 
groups used for each work sample are listed below: 



Sample sizes for each group range from 50 to over 500. All samples are 
clearly described. Means and standard deviations are given for time, error 
and performance scores for each group. Results are given in percentage 
performance scores for each group. 

b. Reliability - The test-retest reliability for each part of each work sample is 
giv.5n. The standard error of measurement was also computed. The reliability 
coefficients were generally very high. Because the methods used to gather 
and analyze the data are not given, no assessment can be made about the 
meaning of these data. 

c. Validity - Each manual contains short descriptions of the diffeifnt types of 
validity. However, no data are available. 



Norm Group 



Work Sample 



(1) 
(2) 
(3) 
(4) 
(5) 
(6) 
(7) 
(8) 
(9) 
(10) 
(11) 



Adventitiously blind 
Air Force 

Community College 

Congenitally blind 

Congenitally deaf 

Employed Workers 

Independent Living 

Methods Time Measurement-MTM 

Sheltered Living 

Skill Center 

Unselected Employed 



1,2, 4, 8, 9, 10, 18 

2, 3, 6, 7, 8, 10, 11 

2, 3, 4, 6, 7, 8, 9, 10, 11 

1,2, 4, 8, 9, 10, 18 

1 through 12 

2, 4, 6, 7, 9 & 1 1 

1, 2, 4, 6, 7, 8 & 11 

1 through 19 

1, 2, 4, 6, 7, 8 & 11 

1 through 12 
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Reviewer's Summary and Comments 



The Valpar Component Work Sample Series currently consists of 19 individual 
work samples that are physically well designed and constructed. They are appeal- 
ing to clients and students and lend themselves to easy administration and scor- 
ing. Individual work samples can be easily incorporated into an existing evalua- 
tion or assessment program. Because the individual work samples can be pur- 
chased as needed by facilities or schools, there are no unified final report forms; 
other aspects of an integrated system are lacking. 

One major problem with the Valpar series is in their relationship to jobs. Ac- 
cording to the manuals, each component is keyed to a number of specific occupa- 
tions as well as Worker Trait Groups. However, the manuals offer no convincing 
evidence that, for example, one work sample in related to ten Worker Trait 
Groups. A second problem is syith the unknown validity of the series. Although 
very well designed, the series offers no objective evidence of having either con- 
struct, content, or empirical validity. 

13. Address 

Valpar International Corp. 
P.O. Box 5767 

Tucson, Arizona 85703-5767 

14. Cast 



Component Work Sample Cost 

Small Tools (Mechanical) $995.00 

Sl7e Discrimination $895.00 

Numerical Sorting $875.00 

Upper Extremity Range of Motion $975.00 

Clerical Comprehensive and Aptitude $1,775.00 

Independent Problem Solving $995.00 

Multi-Level Sorting $1,125.00 

Simulated Assembly $1,375.00 

Whole Body Range of Motion $i, 295.00 

Tri-Level Measu^-ement $1,350.00 

Eye-Hand-Foot Coordination $1,125.00 

Soldering and Inspection $1,125.00 

Money Handling , $1,075.00 

Integrated Peer Performance $2,095.00 

Electric Circuitry and Print Reading $1,095.00 

Drafting $995.00 

Complete set of the above 16 work samples $19,165.00 

CUBE $3,295.00 

Dynamic Physical Capacities $1,595.00 

Adaptation kits to use 1, 2, 4. 8, 9 & 10 

with visually impaired $225.00 each 




146 



14 



5 



Brandon, T. L., Button, W. L., Rastatter, C, J., & Ross, D. R. (1975). Valpar 
Component Work Sample System. Vocational Evaluation and Work Adjustment 
Bulletin. &(2), 59-63. 

Grov/ick, B., Kaliope, G., & Jones, C. (1983). Sample norms for the hearing- 
impaired on select components of the Valpar Work Sample Series. Vocational 
Evaluation and Work Adjustment Bulletin. 16(2), 56-57. 

Jones, C, & Lasiter, C. (1977). Worker-non-worker differences on three Valpar 
Component Work Samples. Vocational Evaluation and Work Adjustment Bul- 
letin. 10(3), 23-27. 

Loera, P. (1977). Vocational evaluation at a resident school for the deaf. Val- 
par-Soective. 2(3), 5-7. 

McCarron, L., et al. A neuromuscular endurance profile and vocational behaviors. 
Denton, TX: North Texas State University, Center for Rehabilitation Studies. 

McEwan, G. (1975). Valpar goes mobile in Arizona. Valoar-Sipective. 2, 7-8. 

Sankovsky, R. (1975). Evaluating rehabilitation potential of the severely handi- 
gapped: Vocationailv related components. Institute, V/V: West Virginia Reha- 
bilitation Center, Research and Training Center. 

Saxon, J. P., et al. (19S3). Intercorrelations of selected Valpar component work 
samples and General Aptituu*, Test Battery score. Vocational Evaluation and 
Wo rk Adjustment Bulletin. >, 20-23. 

Va?par-Spective. (1975; ...On the making cf behavioral observations. Valoar-Soec- 
Liv^. i(4), 1-3. 

Valpar-Spective. (1977). Methods time meacuremcat - another tool for the VCWS 
practitioner. Valnar-Soective. 2(1), 5-6. 



146 



Vocational Evaluation System by Singer 

(VES, New Concepts Corp./Singer Career System?; or Singer) 



1. Development 

a. Sponsor - The system was developed by the Sinjjer Educational Division, 
Career Systems. In October, 1985 the Career Systems products line, its 
patents and copyrights were purchased by the New Concepts Corporation. 

b. Target Group - According to the manual, "The VES is primarily intended for 
specific needs populations (e.g., socially and educationally disadvantaged, 
mildly retarded, physically handicapped) but may also be used with essentially 
normal populations. Those special needs groups who have limited reading 
ability, test poorly, and have a lack of occupational experiences..." In ad- 
dition, many of the work samples can be adapted for deaf persons and for 
Spanish speakers. Thus, it appears that the Singer developers feel the system 
could be used with a wide range of rehabilitation, educational, and manpower 
populations. 

c. Basis of the System - Each work sample is based on a group of tasks con- 
tained in closely related jobs. The basis is a combination of job analysis 
procedures and the job descriptions contained in the fourth edition of the 
Dictionary of QccUDational Titles. 

2. Organization 

a. Name and Number of Work Samples - Presently the following 28 systems arc 
available: 

(1) Sample Making - Measures, cuts, drill, files, and finishes a ring. 

(2) Bench Assembly - Attaches screws and bolts of various sizes to a metal 
block. 

(3) Drafting - Draws concentric circles, produces isometric drawing, and 
draws cut dado block. 

(4) Electrical Wiring - Makes, solders, and tapes three different splices. 

(5) Plumbing and Pipe Fitting - Measures, cuts and reams galvanized and 
plastic pipe; fills out job order; installs a P-trap; assembles water lines; 
and repairs a faucet. 

(6) Woodworking - Constructs a shelf using hand and power tools. 

(7) Refrigeration and Air Conditioning - Measures, cuts and reams copper 
tubing; solders fittings; and follows simple service manual. 

(8) Sales Processing - Completes a credit card sale; and takes and prepares 
telephone orders. 

(9) Needle Trades - Threads and operates a power sewing machine to make 
a bag. 

(10) Masonry - Mixes mortar and lays a corner of bricks. 

(11) Sheet Metal - Lays out; cuts and bends using box and pan break; drills; 
and assembles a box. 

(12) Cook and Baker - Follows directions to prepare brownies in a micro- 
wave. 

(13) Engine Service - Disassembles engine and checks parts; changes oil; 
adjusts points and spark plug; and reassembles engine. 
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(14) Medical Service - Applies elastic bandage to artificial arm; takes own 
temperature, pulse and respiration rate; determines liquid intake and 
output; and tests urine for sugar content. 

(15) Cosmetology - Cuts hair on a mannequin, rolls and pins hair; and 
shampoos, combs, and dries hair. 

(16) Data Calculation and Recording - Uses a calculator to compute the 
payroll and reconcile a quarterly report for 13 employees. 

(17) Production Machine Operating - Operates a simple injection molding 
machine to produce checkers. 

(18) Household and Industrial Wiring - Wires a light fixture, switch and 
outlet into a circuit; and attaches three-prong plug. 

(19) Filing, Shipping and Receiving - Files 100 cards alphabetically and 
numerically; compares purchase and packing slips; compares receiving 
and incoming orders; prepares packages for shipment; and files purchase 
orders. 

(20) Packaging and Materials Handling - Seals and packs boxes; wraps 
fragile items; bands cartons; and moves cartons. 

(21) Electronics Assembly - Constructs a wiring harness; selects and mounts 
components onto a printed circuit board; solders components and wiring 
harness to board. 

(22) Welding and Brazing - Runs puddles and beads; and joins two pieces of 
metal using the outside corner and butt weld with a gas torch. 

(23) Office Services - Takes, records and files simulated telephone messages; 
files letters and invoices; checks and proofreads; sorts incoming mail; 
and uses an electric typewriter. 

(24) Basic Laboratory Analysis - Determines acidity, nitrogen, phosphorous, 
potash and pH of soil samples. 

(25) Diesel Engine Service - Makes minor adjustments and services on a 
small diesel engine. 

(26) Auto Body Repair - Repairs a dented fender; mixes and applies body 
putty; and sands and primes fender. 

(27) Machine Trades - Scts-up, adjusts, and operates a small drill press and 
lathe. 

(28) Information Processing - Set-ups and operates a microcomputer; com- 
pletes letter and number matching activities; and posts information from 
business documents. 

b. Grouping of Work Samples - Each work station is independent. 

c. Manual - Four basic manuals come with the work sample stations: Adminis- 
tration Manual, Technical Manual, Installation Manual and Behavioral Obser- 
vation/Report Writing Manual. These manuals are supported by an audiovisual 
film strip series that enables the user to visually review each manual prior to 
the installation of the work sample stations. Each manual is very complete, 
well-planned and highly organized. 

3. Physical Aspects 

a. Packaging of the Work Samples - Each work station is self-contained in a 
carrel that can be closed and locked when not in use. Some larger pieces of 
equipment, such as the box-and-pan break and microwave ov^cn are located 
outside the carrels. 
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b. Durability - Because the Singer stations use fairly sophisticated tools and 
equipment, it is expected that there would be some problems with durability. 

c. Expendable Supplies - Many of the stations use a considerable amount of 
expendable supplies. For example, the Sheet Metal station requires two fairly 
large squares of metal, hinges, a latch and pop rivets. Other stations require 
wood, wire, fabric, and baking ingredients. Basei on Singer price lists of 
January, 1984, the average price of expendable supplies is $3.52 per work 
sample. The range is from $0.33 to $9.48. 

d. Replacement - All expendable supplies as well as most of the tools can be 
locally obtained. Other parts can be ordered from New Concepts Corp. 

e. Computer Requirements - The Information Processing work sample requ* :s 
the use of a computer. However, because there are no computer requirements 
for overall system use, this section is not relevant 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The order and number of the work 
stations administered is left to the discretion of the evaluator. 

c. Client Involvement - The client is involved in the evaluation process through 
a series of self-ratings on interest and performance. Due to the frequent 
evaluator checkpoints in each work sample, the possibility for client contact 
with the evaluator is moderate to high. The manual does not specify if 
formal feedback is to be given to the client at the end of the evaluation 
process. 

d. Evaluation Setting - The use of carrels and audiovisual instructions creates a 
school-like atmosphere. 

e. Time to Complete the Entire System - The manual states that "a general rule 
of thumb is to allow two to two and one-half hours per job sample." Because 
any number of stations may be administered, no realistic estimates of the 
time to complete the total system can be given. 

Administration 

a. Procedures - The tools and materials needed as well as set-up and main- 
tenance are given in the manual for each work station. All client instruc- 
tions are given on Auto-Vance equipment. 

b. Method of Instruction Giving - All instructions are given using an audio- 
cassette tape and filmstrip format with the client controlling the rate of 
advancement. Typically, the client or student hears several frames of in- 
struction, then turns off the equipment, performs a specific task and then 
calls the evaluator to check that task. The linear programmed material is 
occasionally supplemented with written instructions. Additional evaluator 
instructions are discouraged because they would interfere with the standard- 
ization; evaluators are to record any type of re-instruction. 
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c. Separation of Learning/Performance - There is very little separation of learn- 
ing from performance in the Singer system. Most student instructions do not 
require a criteria to be reached before going on to repeat the task on a 
timed basis. On four of the stations, the client completes a product, has it 
checked by the cvaluator, and then performs the task again without instruc- 
tions on a speeded basis. 

d. Providing Assistance to the Client - The evaluator is encouraged to make sure 
that the student or client knows how to do the task before he/she begins to 
work. Checkpoints are provided in the audiovisual material so that the client 
can ask the cvaluator to review his/her progress before continuing. 

e. Repeating Work Samples - Work samples may be repe^^ted at the "request of 
the client who expresses a desire to try to improve his or her performance." 
The teacher or cvaluator may have a work station repcateJ to assess changes 
in performance. 

Scoring and Norms 

a. Timing - The client is timed by the cvaluator. For each work station, the 
manual contains instructions at the frame numbers where the evaluator is to 
start and stop timing. 

b. Timing Interval - This interval varies with each work sample and is specified 
for each work sample in the manual. In many work samples, there are sev- 
eral timing intervals. 

c. Time Norms - All norms arc based on the number of minutes to complete the 
work sample. Participant and/or employed worker norms are reported using a 
five-point rating scale based on the time score distribution foi each work 
sample. Methods Time Measurement (MTM) norms arc reported in 10% ir Ner- 
vals, with the industrial normal being 100%. 

d. Error Scoring - All errors are carefully defined and each item (or the entire 
finished product) is checked against criteria. In using MTM J.tandards for 
quality, each error is classified as major, intermediate, or minor. 

c. Scoring Aids - Some use is made of scoring aids. 

f. Quality Norms - Participant norms arc reported using a five-point rating 
scale. Industrial norms and MTM quality norms are based on 100% with a 
specified number of percentage points subtracted for each of the three error 
levels. 

g. Emphasis in Scoring - Time and errors arc given equal weight. 
Observation of Clients 

a. Work Performance - Twenty work factors (e.g., attention span, form dis- 
crimination, neatness, and use of hand tools) are defined. Each work sample 
has a separate Task Observation Record containing specific factors for each 
task. For example, in the part of the Drafting Work Sample that "compares 
drawings with models provided,** the evaluator is to observe **foi.<owing a 
model, inspection and checking, retention.** 
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b. Work Behaviors - No work behaviors are listed. 

c. Recording System - Work performance factors are listed on the Task Observa- 
tion Record; the evaluator does not rate behaviors, he/she records the obser- 
vations. A Work Activity Rating Form is used by the client to rate his/her 
interests in a work station before and after performing the tasks; the client 
and evaliiator also rate the client's performance on a five-point scale at the 
end of the work sample. This rating is general and does not include separate 
ratings for work factors. 

d. Frequency of Observation - The manual lists frequent evaluator check and 
assistance points to allow the evaluator to make numerous observations at 
each station. 

8. Reportinn 

a. Forms - Forms include the Task Observation Record, Work Activity Rating 
Form, MTM Rating Form, Industrial Rating Form and a summary sheet for 
time and quality scores. 

b. Final Report Format - While the manual does not contain any recommended 
final report format, it does contain a description of what should be contained 
in a final report. 

9. Utility 

a. Vocational Exploration - An extensive amount of occupational information is 
provided the student or client. Each work sample contains an introduction to 
some jobs related to that work station. Many schools and rehabilitation 
facilities use the Singer primarily as ?,n interest and career exploration de- 
vice. 

b. Vocational Recommendations - Because the system contains no final report 
format, these would be dependent on the user. 

c. Counselor Utilization - For the reason given immediately above, this can be 
judged. 

10. Training in the System 

a. Training Required - No. 

b. Training Available - Yes. 

c. Duration - Two day, one week or two week VES workshops are offered on a 
regional level on a fee basis. 

d. Follow-up - New Concepts regional managers conduct follow-up visits with 
charge. Technical consultations may be scheduled through the home office. 
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11. Technical Considerations 



a. Norm Base - Each unit contains three types of norms: participant or client, 
employer worker, and MTM Many norm groups are of adequate size; sample 
characteristics are thoroughly described. 

b. Reliability - A study by Cohen and Drugo (1976) reported test-retest relia- 
bility coefficients of .61 and .71 for an EMR population. 

c. Validity - VES bases its validity on several sources. First, the content 
validity of the job-task matrix and of the job analysis for each work sample. 
The job-task matrix relates specific tasks to specific jobs and identifies which 
tasks are included in the work sample. The average work station covers 
about 65% of the tasks given in the matrix. Second, two predictive studies 
(Gannaway and Sink, 1972; Monroe County, n.d.) attempted to relate work 
sample scores with success in jobs related to the work sa^- pies. While these 
studies have methodological problems, the significant results are encouraging. 
Third, a study by Sink et al. (1976) revealed that the system encouraged users 
to seek additional occupational information. 

12. Reviewer's Summary and Comments 

The VES or Singer is intended for use mostly as an occupational exploration 
system. Instead of attempting to produce work samples directly related only to 
one job, each work station relates to several jobs having common tasks. This 
results in work samples that are much more versatile, especially for occupational 
exploration. Each station includes several visuals showing the relationship of the 
work station to many specific jobs. 

Physically the stations are well-designed. They are self-contained and the audio- 
visual instructions move the client or student through each work sanple at a slow 
pace. The tasks comprising each work sample offer a variety of experiences. 
Finally, the documentation for the system is thorough. 

Although there needs to be additional validation studies, the ones cited above are 
a small beginning. The major problems are the lack of a work atmosphere, which 
cannot help but detract from the occupational exploration purpose of the system, 
the amount of expendable supplies, and the possible need for a general framework 
to integrate the individual units into one system. 

13. Address 

For genera! information contact thr. Marketing Division at: 

New Concepts Corporation 
1 161 N. El Dorado Place 
Suite 343 

Tucson, Arizona 85715 

For ordering and technical support contact: 

New Concepts Corporation 
1802 N. Division Street 
Morris, Illinois 60450 
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14. Cost 



The cost per work station ranges from $1,659.00 to $3,439.00. The average cost is 
$2,104.00. All prices are F.O.B. Morris, IL. Each work station includes enough 
supplies to evaluate approximately 30 people. 
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Vocational information and Evaluation Work Samples 



(VIEWS) 



1. Developmen t 

a. Sponsor - The VIEWS was developed oy the Vocational Research Institute of 
the Philadelphia Jewish Employment and Vocational Service. 

b. Target Group - The system is especially designed for moderate and severely 
mentally retarded adults. 

c. Basis of the System - The VIEWS is based on six Worker Skill Groups falling 
withip Tour Data, People, Things, (DPT), levels of the fourth edition of the 
Dictio n g v of Occupational Titles. These levels were chosen because they 
represent the most common areas of training and employment for mentally 
retarded persons. 

2. Organization 

a. Name and Number of Work Samples - The 16 work samples are organized 
according to Worker Skill Group.^^ These groups are Vocational Research 
Institute construct? which represent a class of exercises involving similar task 
demands. 

(1) .687 - Materials Sorting includes: #1 Title Sorting, #2 Nui3, Bolts & 
Washers Sorting, #8 Valve Disassembly; Clerical Matching & Counting 
includes: #5 Stamping, #10 Mail Sort, #11 Mail Count; Assembling 
includes: #4 Collating & Stapling, #6 Nut Weighing, #7 Nut, Bolt & 
Washers Assembly, #9 Screen Assembly. 

(2) .686 - Machine Feeding - #12 Machine Feeding 

(3) .685 - Routine Tending - #3 Paper Cutting, #16 Drill Press 

(4) .684 - Fabricating - #13 Budgette Assembly, #14 Valve Assembly, #15 
Circuit Board Assembly 

b. Grouping of Work Samples - The work sampler are grouped according to the 
DPT levels listed above. 

c. Manual - The manual contains the following information for each work sam- 
ple: demonstration, setup, training, production and norms. A photograph is 
used for each work sample to insure proper setup. The use of the recording 
forms, report forms, etc., is not covered in the manual. These are dealt with 
during training. 

3. Physical Aspects 

a. Packaging of the Work Samples - Fourteen work samples are individually 
packaged in portable plastic cabinets. The Drill Press and Machine Feeding 
Work Samples are permanently mounted on a sturdy worktable. 



As in the DOT, the first d;git beyond the decimal point represents Data level, 
the second People, and the third Things. 
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b. Durability - The plastic cabinets as well as the components for each work 
sample appear to be very durable. 

c. Expendable Supplies - Three different colors of 8 1/2 by 11 paper, string, and 
fibcrboard squares are the only expendable supplies used. These all can be 
purchased locally. 

d. Replacement • In order to insure standardization, all replacement parts should 
be ordered from the developer. 

e. Computer Requirements - Not applicable. 
Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The work samples are given from 
least complex to most complex. Each work sample has three phases: (1) 
Demonstration - the evaluator follows the manual to provide an oral descrip- 
tion and a physical demonstration for the client; (2) Training - the client is 
trained to a predetermined criterion of mastery on each work sample— during 
this phase the evaluator is free to use a wide variety of techniques to make 
certain that the client learns the task; and (3) Production - after the crite- 
rion have been achieved, the client is assigned a set number of cycles of the 
work sample to perform independently. The purpose in separating the train- 
ing and production phases is to make sure that the client has learned each 
task before he perfo.iAS it. 

c. Client Involvement - There is extensive client involvement. In the training 
phase for each work sample, the evaluator and the client have a significant 
amount of interaction during the learning process. The Evaluator 's Handbook 
calls for an informal client feedback session after the first day as well as on 
subsequent days when needed. 

d. Evaluation Setting - A realistic work atmosphere aud setting are stressed in 
the Handbook and during evaluator training. 

e. Time to Complete the Entire System - The developer estimates that the 
VIEWS can be administered in from four to seven, five hour days (i.e., 20 to 
35 hours). 

Administration 

a. Procedures - The Handbook contains all details necessary for administration. 
A photograph of each work sample is used to insure proper layout. The 
instructions for the demonstration phase are given in detail and include both 
oral and physical directions. The training phase criteria are clearly given. 

b. Method of Instruction Giving - No reading is required of the client for any 
work sample. The demonstration phase uses oral instructions plus modeling. 
During the training phase the evaluator is free to use a variety of verbal and 
nonverbal techniques; flexibility is stressed here. Because each work sample 
is individually administered, the client can receive instructions using the 
methods which ^est meet his needs. 
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c. Separation of Learning/Performance - As stated above, the VIEWS separates 
learning and performance by having a formal training period for each work 
sample. Here the evaluator is free to use almost any teaching technique that 
will result in the client reaching the established criteria. For example, the 
criterion for the Valve Disassembly Work Sample is: "Two valves consecu- 
tively disassembled and sorted without error." 

d. Providing Assistance to the Client - Extensive assistance is provided during 
the training phase; little is given during the production phase. If help is 
needed during the production phases, the evaluator is to record this on the 
appropriate behavior observation form. 

e. Repeating Work Samples - The VIEWS does not place much emphasis on 
repeating work samples; it is designed so that the client should have learned 
the task before the performance phase. However, work samples may be 
repeated if considered necessary by the evaluator. 

Scoring and Norms 

a. Timing - The evaluator use a time stamp machine to time the client. 

b. Timing Interval - Timing on each work sample begins when the client enters 
the production phase and ends with the completion of the task. 

c. Time Norms - Time results are rated on a three-point scale based on the 
number of minutes to completion. Predetermined time standards using the 
MODAPTS approach are also available. 

d. Error Scoring - Each work sample is checked against carefully defined quality 
standards. No random check is made; the entire work sample is scored. 

e. Scoring Aids - Some use is made of scoring aids. 

f. Quality Norms - The total number of errors for each work sample are con- 
verted to a three-point rating scale. The system also contains rate-of-learn- 
ing norms for use during the training phase. 

g. Emphasis in Scoring - Time and quality are both given equal weight in the 
VIEWS. 

Observation of Clients 

a. Work Performance - Ten work performance factors (e.g., color discrimination, 
finger dexterity, work rhythm) are carefully defined. In addition to these 
definitions, specific definitions of the factors are made for each work sample. 
For example, in the Title Sorting Work Sample, finger dexterity is assessed by 
"picking up tiles with fingers," in the Stamping Work Sample, it is "turning 
pages; picking up stamps." Each work sample has several factors listed that 
are to be observed. The accurate recording of behavioral observations is 
emphasized. 

b. Work Behaviors - ork behaviors such as attendance and punctuality, re- 
sponse to training, and communication are clearly defined and observed during 
the course of the day. 
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c. Rccofuing System - Work behaviors anu performance factors are written, as 
they are observed, on a client record form. Specific behaviors are reported 
on the forms; no rating system is used. 

d. Frequency of Observation - The VIEWS uses extensive observations. Observa- 
tion of defined work factors is required for each work sample. Work be- 
havior observations are to be made constantly. 



8. Reporting 



a. Forms - The system uses four types of standardized forms: (1) a client 
record form for recording training observations, performance observations, 
behavioral observations, and errors (there is a separate page of this form for 
each work sample); (2) a daily observation form for summarizing work be- 
haviors and performances; (3) a final report form, and (4) a profile sheet. 

b. Final Report Format - The VIEWS final report uses a standardized format to 
present information on the following: general observations, interpersonal 
relations, training, worker characteristics, recommendations, and a profile 
sheet containing work sample results including the industrial time standards 
for the work samples. Recommendations are given for training techniques. 
Work Groups, and for other services that may be required. 

9. Utility 

a. Vocational Exploration - Since the tasks are work samples and not actual jobs 
and because almost no occupational information is provided, the VIEWS is of 
little use in occupational exploration. 

b. Vocational Recommendations - Specific recommendations are made; these are 
related to the six Worker Skill Groups and from DPT levels covered by the 
VIEWS. 

c. Counselor Utilization - The system and the final report are oriented toward 
the counselor. 

10. Training Required 

a. Training Required - Yes 

b. Training Available - Yes 

c. Duration - One week in Philadelphia for new users. Under certain conditions, 
regional training is available. 

d. Follow-up - Not available 

1 1. Technical Considerations 

a. Norm Base - The VIEWS was renormed in 1979 on 452 mentally retarded 
persons (mean. IQ = 53) between the ages of 15 and 61. All data are repor- 
ted only in terms of the 1-2-3 ratings. No means and standard deviations are 
given for the time and error scores for any of the work samples. MODAPT*S 
predetermined time standard norms are also available. 
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b. Reliability - No data presently available. 

c. Validity - No data presently available. 

12. Reviewer's Summary and Comments 

The VIEWS attempts to evaluate the vocational potential of mentally retarded 
adults for jobs in four DPT levels. The system relates to job areas that are very 
common in the national economy and more important to the job areas where many 
retarded persons have found successful employment. The most unique feature of 
the system is the attempt to separate learning from performance. The developers 
believe that the client should first be thoroughly taught the task prior to per- 
forming it under timed conditions. The VIEWS also uses standardized behavior 
observations which are combined with time and quality scores to produce a well 
organized final report. One problem with using the VIEWS by itself is the lack 
of occupational information. The system has unknown reliability and validity; 
these are the most critical problems with the VIEWS. 

13. Address 

Vocational Research Institute 
Jewish Employment and Vocational Service 
2100 Arch Street, 6th Floor 
Philadelphia, Pennsylvania 19103 

14. Cost 

$9,950.00 includes: work samples, manuals, forms, and tuition for training two 
persons in Philadelphia. Living expenses and transportation are not included in 
the price. 

15. References 
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Vocational! Interest Temperament and Aptitude System 



(VITAS) 



Development 

a. Sponsor - The VITAS was developed by the Philadelphia Jewish Employment 
and Vocational Service under contract with the Manpower Administration. 
The system was originally designed for use within the U.S. Employment Ser- 
vice. JEVS is presently marketing the system to schools, rehabilitation cen- 
ters and manpower programs. 

b. Target Group - According to the manual, "VITAS is designed for educationally 
and/or culturally disadvantaged persons of both sexes. The system is not 
intended for individuals with more than a 12th grade education, the physically 
handicapped, or the mentally retarded." This reviewer, however, believes that 
the VITAS could be used with many physically handicapped persons and mildly 
retarded persons. 

c. Basis of the System - The VITAS is based on 16 Work Groups in the Guide 
for Occupational Explorati on (GOE). These were .selected because of employ- 
ment and/or training opporttjnities. 

Organization 

a. Name and Number of Work Samples - The 16 Work Groups are assessed by 21 
separate work samples listed below. (Note that several of the work samples 
assess for more than one work group.) 

(1) 02.04 - Laboratory Technology - #4 Collating Material Samples, #8 Nail 
& Screw Sorting (Part I); #8 Nai! & Screw Sorting (Part II); and #20 
Laboratory Assistant 

(2) 05.03 - Engineering Technology - #21 Drafting 

(3) 05.05 - Craft Technology - #11 Lock Assembly, #19 Spot Welding and 
#21 Drafting 

(4) 05.09 - Materials Control - #2 Packing Matchbooks, #3 Title Sorting 
and Weighing, #5 Verifying Numbers, #8 Nail & Screw Sorting (Part II) 

(5) 05.10 - Crafts - #7 Budget Book Assembly, #8 Nail & Screw Sorting 
(Part I), #9 Pipe Assembly, #11 Lock Assembly 

(6) 05.12 - Elemental Work: Mechanical - #1 Nuts, Bolts & Washers 
Assembly, #2 Packing Matchbooks and #6 Pressing Linens 

(7) 06.01 - Production Technology - #8 Nail & Screw Sorting (Part I), #11 
Lock Assembly, #12 Circuit Board Inspection, #19 Spot Welding, #20 
Laboratory Assistant, and #21 Drafting 

(8) 06.02 - Production Work - #1 Nuts, Bolts & Washers Assembly, #6 Pres- 
sing Linens, #7 Budget Book Assembly, and #9 Pipe Assembly 

(9) 06.03 - Quality Control - #3 Tile Sorting and Weighing, #4 Collating 
Material Samples, #8 Nail & Screw Sorting (Part I), #12 Circuit Board 
Inspection 

(10) 06.04 - Elemental Work: Industrial - #1 Nuts, Bolts & Washers Assem- 
bly, #2 Packing Matchbooks, and #6 Pressing Linens 

(11) 07.02 - Mathematical Details - #8 Nail & Screw Sorting (Part II), #13 
Calculating, #15 Bank Teller, and #17 Payroll Computation 
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(12) 07.03 - Financial Detail - #8 Nail & Screw Sorting (Part 11), #13 Cal- 
culating, #15 Bank Teller, #17 Payroll Computation, and #18 Census 
Interviewing 

(13) 07.04 - Oral Communications - #14 Message Taking and #18 Census 
Interviewing 

(14) 07.05 - Records Processing - #5 Verifying Numbers, #10 Filing by Let- 
ters, and #16 Proofreading 

(15) 07.06 - Clerical Machine Operation - #13 Calculating and #14 Message 
Taking 

(16) 07.07 - Clerical Handling - #2 Packing Matchbooks, #3 Tile Sorting and 
Weighing, #10 Filing by Letters 

b. Grouping of Work Sarxjples - Each work sample is independent in terms of 
administration and scoring. The results are combined and interpreted as part 
of the above listed classifications. Work samples are also related to work 
groups. 

c. Manual - The manual contains the following information on each work ^.ampler 
(1) inventory; (2) administration notes; (3) demonstration/instructions, and (4) 
scoring procedures. A photograph i: used for each work sample to insure 
proper setup. Although examples of report forms and definitions of the 
aptitude codes are given in the manual, there are no instructions on how to 
use these items. These are covered during training. 

Physical Aspects 

a. Packaging of the Work Samples - All work samples are packaged independent- 
ly. Most of the work samples are packaged in plastic containers that could 
easily be stored when noi la use. The only heavy piece of equipment is the 
spot welder. 

b. Durability - The VITAS uses mostly basic tools and equipment that should be 
very durable. According to the developers, no problems have been reported. 

c. Expendable Supplies - The VITAS uses expendable items such as: paper and 
recording forms, string, and sheet metal All supplies can be locally obtained. 
While there are no estimates, the cost per client administration is low. 

d. Replacement - All replacement parts should be ordered from the developer in 
order to maintain standardization. 

e. Computer Requirements - Not applicable 
Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is necessary. 

b. Sequence of Work Sample Administration - The client usually begins with the 
least complex work sample (i.e.. Nuts, Bolts and Washers Assembly) and pro- 
gresses to the more complex (i.e.. Drafting). However, the work samples can 
be given in any order. 

c. Client Involvement - The client is involved in the vocational process at 
several different times: (1) new clients are given an orientation se<5Sion when 
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first coming into the evaluation unit; (2) a group motivational session at the 
end of the first day of evaluation; and (3) a feedback and interest interview 
after the work samples arc administered. 



d. Evaluation Setting - The VITAS manual stresses a realistic work setting. 

e. Time to Complete the Entire System - According to the manual, "most clients 
can complete the work sample within three, five-hour days," or about 1 5 
hours. 

Administration 

a. Procedures - A photograph showing the correct arrangement of parts is given 
to insure proper setup of each work sample. The instructions for the demon- 
stration part of the work sample are brief and to the point; these include 
both oral and physical directions. For each work sample, a section called 
"Administration Notes*" contains additional items for setup and any special 
instructions that are to be followed by the evaluator during thvC client demon- 
stration. 

b. Method of Instruction Giving - Client instructions are given orally in all 
cases and in some cases by demonstration. While the client is not required 
to read any administration instructions, reading and the use of mathematical 
skills are needed to successfully complete six of the work samples. 

c. Separation of Learning/Performance - Nine of the VITAS work samples do not 
have a separate practice period. After the evaluation instructions and demc n- 
strations are completed and any client questions answered, the client begins 
the task. No criteria are used to establish that the client has learned the 
task. For all practical purposes, there is no separation of learning from 
performance. 

d. Providing Assistance to the Client - "When a client has a question or seeks 
assistance, the evaluator may repeat the necessary part of the instructions, 
including a re-demonstration, but should never do part of the task for the 
client. Only a minimum amount of assistance should be given so as to en- 
courage the clients to do as much as they can on their own." Thus, when 
administering a work sample, the evaluator gives the instructions and demon- 
stration and does not offer any additional explanations unless requested. 

e. Repeating >Vork Samples - Re-administration of work samples is not recom- 
mended. Ho\vever, clients are urged to complete a task as best they can once 
they begin. 

Scoring and Norms 

a. Timing - The evaluator uses a time stamp machine to mark the starting time 
on a slip of paper; at the completion of the >v^ork sample, the client stamps 
his/her own slip. 

b. Timing Interval - Timing begins when the evaluator completes the instruction 
phase of the work sample and ends when the client completes the task. 
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c. Time Norms - The minutes to completion are converted to 1-2-3 rating. No 
percentile or standard scores are used. 

d. Error Scoring - All v/ork samples have carefully defined scoring procedures in 
which errors arc clearly described (e.g., "Number of books with string not 
tied in a bow.") No random samples are used; the entire work sample is 
checked. The frequency of each type of error is recorded. 

e. Scoring Aids - Extensive use is mj^de of scoring aids such as templates, 
overlays, coding systems, and measuring instruments. 

f. Quality Norms - The errors are converted to a three-point quality rating. As 
with the time scores, no percentile or standard scores are used. 

g. Emphasis in Scoring - Both time and errors are given equal weight. 
Observation of Clients 

a. Work Performance - The VITAS manual describes nine work performance 
factors (i.e., aptitudes). The definitions of the factois (e.g.^ spatial, clerical 
perception, color discrimination) are taken from the DOT definitions. Specific 
definitions of each aptitude are related to each appropriate work sample. 
Thus, in the Packing Matchbook Work Sample, Manual Dexterity, Aptitude is 
observed as "Transferring matchbooks from bin to tray; handling trays." 
While in the Tile Sorting and Weighing Work Sample, Aptitude ic observed by 
picking up boxes. Each work sample has from two to seven aptitudes that 
are to be observed. The close and accurate observation of c!«^nt behaviors is 
stressed. 

b. Work Behaviors - General observations are to be made on attendance, punctu- 
ality, verbal ability, interpersonal behavior, and general worker characteris- 
tics. These are to be observed and recorded throughout the working day. 

c. Recording System - No rating or checklist system is used; specific behaviors 
for each work sample are recorded on a separate form for that work sample. 

d. Frequency of Observation - While the manual contains no specific instructions 
as to when and how often to observe behavioi, the system stresses the almost 
constant observation of client behavior. 

Reporting 

a. Forms - The system uses five types of standardized forms* (1) a Work Sam- 
ple Record Form for recording; aptitude and behavior observations, types of 
errors, and time and error scores (there is a separate page of this form for 
each work sample), (2) a general observation form, (3) a final report form, (4) 
a Vocational Interest Interview Form, and (5) a profile sheet. 

b. Final Report Format - A four page fina' report form uses a standardized 
format to present information on the following: Physical description; atten- 
dance and punctuality, verbal ability, interpersonal behavior, skills, vocational 
recommendations by Work Groups, recommendations for supportive services, 
and profile of all work sample time and quality ratings. 



166 



1G3 



9. 



U tility 



a. Vocational Exploration - The VITAS is of limited use for providing the client 
with occupational information. The nature of many of the tasks is abstract 
and no job information is provided during the instruction period for each 
work sample. However, the manual states that occupational/vocational infor- 
mation should be used as a supplement to the VITAS. 

b. Vocational Recommendations • Recommendations are made in two specific 
areas: (1) the most feasible Work Groups for employment or training and (2) 
specific supportive services needed to obtain the employment goal. Apparent- 
ly, recommendations within each work group are kept general - no specific 
jobs are suggested. 

c. Counselor Utilization - The final report is aimed at the counselor who needs 
to make fairly specific vocational decisions. 

10. Training in the System 

a. Training Required - Yes 

b. Training Available - Yes 

c. Duration - A onc-v c k fraining session is held in Philadelphia. Regional 
training is available under certain conditions. 

d. Follow-up - Not available 

11. Technical Considerations 

a. Worm Base - There are two norm groups for the VITAS. The 1980 norms 
contained results on a sample size of about 325 persons. The sample is 
clearly described— 63% female, white 63%, median age - 25 years and 35% 12th 
grade education. The 1981 secondary schrol norms have about 220 cases for 
each work sampK^. The mean age is 15.2 years, white - 7S%, male - 68% and 
67% learning disabled. Time and error scores are converted to a 1-2-3 rating 
system. No employed worker or predetermined time standard norms are given. 

b. Reliability - No data are a-^ilable. 

c. Validity - No data are available. The manual makes reference to "face" 
validity as a criterion and then confuses this with content validity. 

12. Reviewer's Summary and Comments 

The VITAS System is the third work sample system developed by Philauclphia 
JEVS. LiKe the JEVS and VIEWS systems, it stresses the importance of careful 
and accurate behavior observations. The system also uses the work sample to 
work group approach that has served JEVS so well in the past. It must also be 
pointed out that many of the VITAS work samples are refinements and modifica- 
tions of the original JEVS system. While the system could provide accurate 
assessment of CETA populations in a relatively short period of time, it has three 
problems: (1) a lack of client occupational information, (2) the failure to make 
any real distinction between learning and performance, and (3) the laclc of any 
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reported reliability and validity data. The emphasis upon close Client contact, 
careful observations, and the practical reporting format are the three major 
advantages of the system. 



13. Address 

Vocational Research Institute 
Jewish Employment and Vocational Service 
2100 Arch Street, 6th Floor 
Philadelphia, Pennsylvania 19103 

14. 

$9,950.00 includes work samples, manuals, forms, and tuition for training one 
person in Philadelphia. Living expenses and transportation for the evaluator are 
not included in the price. 

15. References 

Abrnms, ^t (1979). A new work sample battery for vocational assessment of the 
disadvantaged VITAS. Vocational Guidance Quarterly, 28(1), 35-43. 

Harris, J. (1980). Final report: VITAS work samples assessment as Part of the 
Job Service demongtration project for Out-of-school Youth. (U.S. DOL con- 
tract #20428020;. 

Sankowsky, R. (1975). Evaluating rehabilitation potential of the severely handi- 
capped: Vocationally related components. Institute, WV: West Virginia 
Rehabilitation Center, Research and Training Center. 

Zimmerman, B. (1979), VITAS. In A. Sax (Ed.), Innovations in Vocational Evalua- 
tion and Work Adjustment. Vocational Evaluation and Work Adjustment 
Bulletin, 12,(1), 29-31. 
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Wide Range Empioyabslity Sample Test 



(WREST) 



1. Development 

a. Sponsor - The WREST was refined and is marketed by Jastak Associates. 

b. Target Group - The original work samples were aimed at supplementing the 
assessment of mentally retarded and physically handicapped persons in a 
sheltered workshop. According to the manual "it is particularly appropriate 
for day activity centers, sheltered workshops, special education facilities, and 
other programs whose participants include mentally retarded, cerebral palsied, 
and other severely physically, mentally, and socially handicapped." Its pri- 
mary use is with persons for whom competitive employment of any kind is in 
doubt. 

c. Basis of the System - The WREST is based on a group of work samples 
originally developed at a sheltered workshop in Wilmington, Delaware for 
"referral of handicapped individuals who may be trained in basic work produc- 
tion skills." The work samples were used in conjunction with other tech- 
niques CO train and select persons for various areas of the workshop. 

2. Organization 

a. Name and Number of Work Samples - There are ten work samples; the first 
two have t^vo parts: 

(1) Folding - Folds a single page and double folds, glues, labels and stuffs 
envelope. 

(2) Stapling - Stapling accuracy and collation and stapling. 

{V; Packaging - Places 8 pegs by color in a small plastic container, 

(4) Measuring - "Filling plastic containers with small grains of colored 
plastic to a specific line." 

(5) Stringing - Runs lacing through a card to form a loop. 

(6) Gluing - Accurately glues paper swatches to lines. 

(7) Collating - "Picking one sheet at a time and placing it in proper 
order." 

(8) Color Matching - Arranges colored swatches to match a sample sheet. 

(9) Pattern Matching - "Placing colored pegs in a block to replicate pat- 
terns" 

(10) Assembling - "Assembling assorted screws, washers, and nuts on a board 
to match a sample.'* 

b. Grouping of Work Samples - Each work sample is independent. 

c. Manual - The single manual contains the following major areas: (1) history, 
(2) theory, (3) general administration guidelines, (4) work sample instructions, 
(5) scoring, (6) technical considerations, and (7) case histories. The general 
administration section is highly detailed as well as useful. The instructions 
for each work sample are well organized and easy to follow. A photograph is 
used for each work sample to insure proper layout. 
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3. Physical Aspects 



a. Packaging of the Work Samples - The work samples, manuals, supplies, and 
scoring forms are shipped in cardboard cartons. 

b. Durability - All work samples are made of heavy (mostly clear) plastic The 
containers should be durable; however, the user should expec. some wear of 
the pegs, tags, and colored pieces. 

c. Expendable Supplies - Besides forms, typing paper, stickers, and colored paper 
swatches are the most common expendable supplies. These are inexpensive 
and locally available. The developer also sells a resupply kit. 

d. Replacement - Replacement parts can be ordered from the developer, 
c. Computer Requirements - Not applicable. 

4. Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The 10 work samples may be 
administered in any order. However, roost evaluators "will find it more 
convenient to follow the designated order." 

c. Client Involvement - The manual stresses that the client(s) should be told 
what the work samples involve and how the results will be used. The need 
for individualized attention is also mentioned. There is, however, no state- 
ment in the manual on providing feedback after specific work samples. 

d. Evaluation Setting - The evaluation setting would most likely be that of a 
formal testing situation. 

e. Time to Complete the Entire System - Administration time for individual 
clients is about one and a ha'f hours; small groups of three to five persons 
take about two iiours. 

5. Administration 

a. Procedures - For each work sample, the manual describes the purpose, and 
gives the materials, scoring information and instrv;tions. A photograph is 
used to ensure proper layout. The WREST can be administered to small 
groups of three to six persons. Duplicate sets of the WREST are nocessar^^ 
for group administration. 

b. Method of Instruction Giving - All instructions are oial and demonstration; no 
reading is required. The manual warns that instructions must be closely 
followed: **any change from the manual may cause confusion, thus invalidat- 
ing the norms of that test." 

c. Separation of Learning/Performance - Each work sample contains a practice 
period prior to the start of timing. While there are criteria for most work 
samples (e.g.. In Assembling, five practice items must be correct), the evalua- 
tor may use additional practice items if necessary to make sure that the 
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client understands the instructions. Thus, there is a separation of learning 
from performance. 

d. Providing Assistance to the Client - The manual clearly states that "once the 
formal testing has begun, no help can be given, but all possible assistance 
should be given during the instruction and practice preceding the formal 
testing." 

e. Repeating Work Samples - Readministration of work samples is emphasized for 
upgrading. Evaluators arc encouraged to keep accurate records of all read- 
ministration. 

Scoring and Norms 

a. Timing - The evaluator times the client(s) using a stopwatch or other timing 
device. 

b. Timing Interval - Timing is started after the client(s) understand(s) the task 
and continues for a set period of time, which is different for each task. The 
time needed to complete each work sample is recorded in minutes and sec- 
onds. 

c. Time Norms - The number of minutes and secondt ;ecorded are compared to 
scaled scores ranging from 0 through 19. The scaled scores can also be 
compared to standard scores. 

d. Error Scoring - All completed parts are checked against the clearly defined 
scoring criteria given in the manual. 

e. Scoring Aids - No use is made of scoring aids. 

f. Quality Norms - The errors for all ten work samples are added together and 
the total compared to norm tables. 

g. Empharis in Scoring - The time results are emphasized. 
Observation of Clients 

a. Work Performance - The manual states that the evaluator should be familiar 
with the dexterity and perceptual aptitudes a. defined in the DOT. However, 
no instructions are given for making, recording, and using these observations. 

b. Work Behaviors - Ten general work behaviors (e.g., appearance, perseverance, 
organization of work, and safety practices) arc defined in the manual. These 
are not defined in behavioral terms. There is a space on the Suhimary Profile 
for rating each behavior category. 

c. Recording Systen* - Each behavioi is rated on a scale from one to 18. Verbal 
descriptions of from "very poor" to "very good" are used in conjunction with 
the numbers. There is no explanation of how these ratings are to be ob- 
tained; there is almost no room on the form for recording actual observations. 

d. Frequency of Observations - This is not specified in the manual. 
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8. 



Reporting 



a. Forms - A single, two page form is used to record all raw scores and to 
report the converted scores, as well as the "behavior" observations. The 
second page of the form contains space for a work history and "summary and 
recommendations." 

b. Final Report Format • A variety of reporting formats are illustrated in the 
manual. These show examples of final reports which incorporate a wide 
variety of data from other sources. The WREST was not intended to be used 
independently of other methods of assessment. 

9. Utilitv 

a. Vocational Exploration - The very simple nature of most of the work samples 
makes the WREST of little use in job exploration for a normal population. 

b. Vocational Recommendations - The manual contains no information on the 
making of vocational recommendations from the work sample results. 

c. Counselor Utilization - The manual contains no information on use of WREST 
results for the counselor. 

10. Training in the System 

a. Training Required - No training is required prior to purchase or use. 

b. Training Available - No formal training is available. 

c. Duration - Not applicable. 

d. Follow-up • Not applicable. 

11. Technical Considerations 

a. Norm Base - Time and quality norms are available on three major groups: (1) 
general population, (2) sheltered workshop employees, and (3) competitively 
employed workers. The general population group is further broken down into 
six age groups and by sex. In the workshop and industrial groups, ages and 
sex were combined when it was discovered that there was little significant 
differences within these general groups. Norm groups are well defined and 
sample sizes lange from 200 for individual groups to 4000 for large groups. 
All samples were collected in the State of Delawaie. 

b. Reliability - Test-retest reliability coefficients for time and error bcores on 
the ten work samples were calculated using 428 employed workers over a 
three month period. All correlations were in the .90's. A second study on a 
very small sample (N=15) repeated the WREST three times over a period of a 
few weeks; the correlations were in the high .80's and low .90's. These 
results are a strong indication of the test-retest reliability of the WREST. 

c. Validity - Validity is based on two correlations between supervisor's ratings 
and time and error standard scores for 428 employed workers. The WREST 
correlated .86 (time) and .92 (quality) with the ratings. These correlations 
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are extremely high; so high in fact that the manual advises that "extreme 
caution must be used in regarding such studies as the above as valid measures 
of test validity." Nevertheless, the results are encouraging. 



12. Reviewer's Summary and Comments 

The WREST consists of ten, short, low-level tasks apparently designed to assess 
mainly the manipulation and dexterity abilities of the client. Although it is not 
stated in the manual, the WREST seems most useful in assessing new clients for 
assignment to suitable work projects within a sheltered workshop. The emphasis 
upon repeating the work samples many times should provide an evaluation of the 
client's ability to improve his performance under repeated practice conditions. 
The major problems of the system center arour.d the lack of systematic behavior 
observations, failure to relate results to the competitive job market, and the 
apparent lack of a usable final report for referring counseloi or agency. Finally, 
the WREST has an adequate norm base, good estimates of test-retest validity, and 
an attempt at establishing concurrent validity. In a field that is all too often 
characterized by poor technical development, the WREST can serve as a good 
example. 

13. Address 

Jastak Associates, Inc. 
1526 Gilpin Avenue 
Wilmington, Delaware 19806 



15. References 

Botterbusch, K. (1973). Wide Range Employment Sample Test. In A. Sax (Ed.), 
Innovations in vocational evaluation and work adjustment. Vocational Evalua- 
tion and Work Adjustment Bulletin, ^(2), 40-43. 

Jastak, J. F., & Jastak, S. (1979). Meanings and measures of mental tests. Wil- 
mington, DE: Jastak Associates. 

Morley, H. (Ec./. (1973). Vocational assessment svotems. Des Moines, I A: State 
of Iowa, Department of Public Instruction. 

Timmerman, W. J., & Doctor, A. C. (1974). Special applications of work evalua- 
tion techniques for prediction of emplovabilitv of the trainaule mentally re- 
tarded. Stryker, OH: Quadco Rehabilitation Center, Inc. 



14. Cost 



Work Sample Kit . 

Manual 

Resupply Kit . . . 
Recoid Forms (50). 



$750.00 
$ 27.50 
$120.00 
$ 10.50 



ERIC 



173 

170 



Work Skiii Development Package 

(WSD) 



1. Development 

a. Sponsor - The WSD was developed by Mr. Donald Bastian of the Attainment 
Company. 

b. Target Group - The pack.ige is designed to develop basic or pre-vocational 
work skills in severely riientally retarded, mentally ill and/or physically dis- 
abled persons. Apparently, the system is intended for use in special educa- 
tion as well as vocational rehabilitation facilities. According to the develop- 
ers, the system has four major applications: (1) the assessment of a baseline 
of performance and of subsequent progress, (2) the acquisition of skills need- 
ed to successfully complete a task, (3) production or refinement of skills with 
emphasis on speed, and (4) the -^velopment of acceptable work behavior ar.d 
work habits. 

c. Basis of the System - The WSD reflects three basic pre-vocational skills: (1) 
ability to discriminate between objects, (2) manipulation of a variety of small 
objects and (3) ability to apply basic concepts. While the WSD is mostly a 
training package for pre-vocational skills, it has limited use as a vocational 
evaluation system. 

2. Organization 

a. Name and Number of Work Samples - The system consists of 20 samples 
arranged in three groups^^: 

(1) Discrimination Tasks - The fCilowing tasks, arranged in ascending order 
of difficulty, are used: (1) Three Item Sort, (2) Basic Size Discrimina- 
tion, (3) Tactile Discrimination, (4) Cue Variable, (5) Subtle Color, (6) 
Subtle Size Sort, (7) Six Item Sort, (8) Rubber Parts Sort and (9) Twen- 
ty-Four Item Sort. These tasks were graded in order of difficulty by 
manipulating three variables: dimensional differences, subtlety of dis- 
crimination, and number of items. 

(z) Assembly Tasks - Each of the following tasks has an acquisition and 
production phase: (1) Tube Assembly/Disascembly, (2) Paint Brush Assem- 
bly/Disassembly, (3) Coupling Assembly/Disassembly, (4) Container Pack- 
aging/Disassembly, (5) Connector Assembly/Disassembly, (6) Pen Assei-n- 
bly/Disassembly, and (7) Shelf Assembly/Disassembly. 

(3) Packaging Tasks - The four Packaging Tasks have corresponding disas- 
sembly tasks: (1) Color Match Sequencing/Disassembly, (2) Snap Box 
Packaging/Disassembly, (3) Small Parts Packaging/Disassembly, and (4) 
Plate Weighing and Bagging/Plate Disassembly. "The Packaging task? 



Assembly/Disassembly and Package/Disassemble units are counted as on^; work 

sample. 
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require the discrimination and manipulation skills of other tasks, but also 
incorporate the application of basic concepts." 

b. Grouping of Work Samples - The work sample tasks are grouped in two 
different ways. The first, by function, is listed above. The second is by one 
of four difficulty levels: 

Series One - Three Item Sort, Basic Size Sort, Tactile Sort, Tube Assem- 
bly/Disassembly, and Color Matching Sequencing. 

Series Two - Cue Variable Sort, Subtle Color Sort, Paint Brush Assembly/Dis- 
assembly, Coupling Assembly/Disassembly, and Snap Box Packaging. 

Series Three - Subtle Size Sort, Six Item Sort, Container Assembly/Disassem- 
bly, Connector Assembly/Disassembly, and Small Parts Packaging. 

Series Four - Rubber Parts Sort, Twenty-Four Item Sort, Pen Assembly/Disas- 
sembly, Shelf Assembly/ Disassembly, and Weighing and Bagging. 

c. Manual - All directions are contained in a single, multi-colored, loose-leaf 
bin'^er. The manual is organized by work sample type (i.e., discrimination, 
assembly and packaging). The following sections are given for each work 
sample: materials, procedure and quality criteria. There are illustrations 
sho'^ing the proper set-up for each task. The administration instructions are 
purposely vague; the trainer is to vary instructions according to the client's 
or student's needs. There is a generalized program plan giving guidelines for 
selecting tasks, selection of goals and reassessment. Some normative data is 
contained in the appendix. 

Physical Aspects 

a. Packaging of the Work Samples - Each work sample is individually packaged 
and contained in one or more bins. No parts are shared between work sam- 
ples. 

b. Durability - The task materials are made from durable metal, wood, plastic 
and rubber. Although WSD materials are not designed for shop or industrial 
use, they are durable for use within a classroom or evaluation laboratory. 

c. Expendable Supplies - Aside from forms and plastic bags for heat sealing, 
there are no expendable supplies. 

d. Replacement - Some replacement parts and supplies are provided in the initial 
package as extras. Other supplies can be ordered from the developer. 

e. Computer Requirements - Not applicable. 
V ocational Evaluation Process 

a. Preliminary Screening - The manual contains no information on preliminary 
screening. Since the purpose of the WSD is to develop some very elementary 
skills, the system is useful as a preliminary screening method for training or 
additional assessment. 
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b. Sequence of Work Sample Administration - The tasks are administered accord- 
ing to the four difficulty Series given in section 2.d. The client repeats each 
task until he/she meets the existing criteria, usually 9C% for accuracy and 
predetermined MTM standard. He/she then progresses to the next task in the 
Series. 

c. Client Involvement - The manual contains no information about client involve- 
ment in the training process and no formal feedback procedures are given. 

d. Evaluation Setting - This is not specified. However, given the nature of the 
tasks, the WSD would fit into both a school classroom and a work activity 
center setting. 

e. Time to Complete the Entire System - Because the system emphasizes training 
to predetermined competency levels, this aspect is not relevant for the WSD. 
Thus, time to complete is dependent on the client's ability and the established 
performance levels. 

Administration 

a. Procedures - All methods, procedu'-es, illustrations arc easy to understand. 

b. Method of Instruction Giving - The evaluator or teacher is urged to be 
flexible and modify instructions, offer examples, give additional cues and even 
change procedures according to individual needs. After the task has been 
selected and any modifications made in the procedures, the teacher selects 
one or more appropriate instructional methods. The ones briefly described in 
the manual are: hand-over-hand physical guidance, shaping, prompting, mod- 
eling, verbal instructions, verbal feedback, self-correction and visual feedback. 

c. Separation of Learning/Performance - The manual clearly divides the ad- 
ministration of all tasks into assessment and training. The training phase is, 
in turn, divided into acquisition and production. Each client/student must 
reach a 90% correct criteria before beginning production. Time and quality 
standards are on^y applied daring the production phase. Thus, the WSD 
clearly differentiates between learning and performance. 

d. Providing Assistance to the Client - The evaluator or teacher is encouraged 
to provide as much assistance as is needed to teach the client the task. No 
mention is made of the type of assistance to be provided during the produc- 
tion phase. 

c. Repeating Work Samples - In the production phase the tasks are repeated for 
the purpose of reaching the exit criteria for each Series of tasks. 

Scoring and Norms 

a. Timing - Exact procedures are not specified. 

b. Timing Interval - Not specified; the manual implies that timing begins after 
the acquisition phase is ended and stops when the last object has been sor- 
ted, assembled, disassembled or packaged. 
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c. Time Norms - MTM norms are provided for the assembly/disassembly and the 
packaging tasks. There are no time norms for the discrimination tasks. 

d. Error Scoring - Each piece is checked against specific criteria for each task. 

e. Scoring Aides - No scoring aides are used. 

f. Quality Norms - There are no quality norms; a percentage of the errors is 
recorded. The manual mentions a 95% accuracy rating for the first three 
Series and a 98% accuracy rating for the fourth Series as being acceptable. 

g. Emphasis in Scoring - Both time and accuracy are equally important. 
Observation of Clients 

a. Work Performance - The system contains a separate Work Performance Data 
Sheet. The following data are recorded for each task: working time, number 
of units, percent of norm, number of errors, percent of accuracy, baseline, 
training, goal and comments/observations. 

b. Work Behaviors - Work behaviors are primarily assessed as on-task or off- 
task. The following off-task behaviors are briefly defined: aggression, work 
station interferences, verbal aggression, inappropriate vocalizations, stereo- 
typic behaviors, out-of-seat behaviors and playing and arranging of work 
materials. The Work Behavior Sheet also contains a space for behavioral 
analysis. 

c. Recording System - There are two recording methods for each W k Behavior 
Data Sheet (Apparently a separate sheet is used for each behavior.): interval 
and event. In interval recording the client is observed at fifteen second 
intervals and the presence or absence of a behavior is recorded. Event 
recording is used for behavior that does not require continuous observation; 
here the behavior is recorded when it occurs during an observation period. 

d. Frequency of Observation - Each observation period has three, five minute 
sessions; each five minute session has 20, fifteen cecond intervals. "An obser- 
vation period consists of three observation sessions, after which you can 
calculate and then note the average occurrence." 

Reporting 

a. Forms - Results and behavior observations are recorded on the Work Behavior 
Sheet and the Work Performance Data Sheet, described above. The WSD also 
includes a Data Display form that permits the graphing of performance by 
time intervals. 

b. Final Report Format - A final report is not mentioned in the manual. How- 
ever,, the Data Display form ccald function in this capacity. 
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utility 



a. Vocational Exploration - 

b. Vocational Recommendations- 

c. Counselor Utilization - 



According to the developers, the 
system is to be used as a "prevoca- 
tional training and assessment pro- 
gram"* to teach ve*y basic skills in 
discrimination, assembly and pack- 
aging. 



Training in the S^St^m 

a. Training Required - Training is required prior to use. 

b. Training Available - In-service training is provided by the developer when the 
system is installed. 

c. Duration - The session is one day in length. 

d. Follow-up - Training and follow-up are included in the price of the package. 
Technical Considerations 

a. Norm Base - MTM norms are available on all assembly and packaging jobs. 
The manual gives both the per unit and per task 100% standard. There is a 
short discussion on how to calculate the client's percentage of industrial 
normal. These substitute for norms tables per se. No quality or error norms 
are available. 

b. Reliability - No studies are reported in the manual. 

c. Validity - No studies are reported in the manual. 
Reviewer's Summary and Comments 

The introduction of the Work Skill Development Package contains the following 
statement: "...The WSD Packsige is not a work evaluation system. It is not 
intended to evaluate or predict vocational readiness through a single admini:,tra- 
tion of tasks." The system i:-^ a prevocational training program intended to be 
part of a skill and "behavior" acqu:sition process. Eecause of this the manual 
assumes that the tasks are administered several different times and for different 
purposes: The first administration is used to assess the client's ability to under- 
stand instructions and perform the task. Second, if the client does not master 
the task, the evaluator or teacher uses various instructional methods to teach the 
skills. The next step is commonly numerous practice sessions to reach a prede- 
termined goal, such as 75% of industrial nownal. Finally, the task can be ad- 
ministered as a sort of post-test. 

The WSD's major purpose is as a training device for mentally retarded, severely 
physically disabled and/or mentally ill persons. The logical progression in com- 
plexity and working with different materials makes the system easy to use with 
moderately and severely mentally retarUsd persons. 

The system's major problem centers around documentation. The manual should 
contain a more complete explanation of the uses on the system. Greater informa- 
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tion is needed on use of the system as an initial assessment device and as a 
training device. The manual lacks data on how to record, plot and use the 
results from repeated task administrations. Because these repeated trials are very 
important in basic skill training, they should be explainec and some examples 
provided. 

13. Address 

Attainment Compai^.y 

P.O. Box 103 

Oregon, Wisconsin 53575 

14. QqsL 

Current price is $2,995.00. This includes shipping, delivery, either a personal or 
video in-service, ar.d some replacement parts. 

15. References 

None presently available. 
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World of Work Inventory 



(WOWI) 



1. Development 

a. Sponsor - WOWI was developed by Dr. Robert E. Ripley and modified by 
Karen S. Hudson and Dr. Gregory P. M Ncidert of World of Work, Inc. 

b. Target Group - According lo its manual, the system is designed for use with 
a wide range of persons: junior and senior high school students; community 
college and college students; adult education; private business and industry 
applicants and trainees; vocational rehabilitation clients; and prisoners. 
Although WOWI can be administered to many types of persons, the instrument 
requires an estimated eighth grade reading level.^^ 

c. Basis of the System - The group of tests, interest inventory, and temperament 
measure arc all related to the fourth edition of the Dictionary 3f Occupation- 
al Titles and the Guide for Occupational Exploration. These two documents 
form the structure of the WOWI. 

2. Organization 

a. Name and Number of Work Samples - The system is composed of a combina- 
tion of either computer administered or paper-and-pencil tests and inven- 
tories. These are as follows: 

(1) Identifying Information - demographical data and best liked school 
subjects arc entered. 

(2) Career Interest Activities - 238 items using a like-neutral-dislike scale 
measure interests in the following 17 Career Families: public service, 
science, engineering and related, business relations, managerial, the arts, 
clerical, sales, service, primary outdoor, processing, machine work, bench 
work, structural work, mechanical and electrical work, graphic arts and 
mining. 

(3) Vocational Training Potentials - The following four alternative, mul- 
tiplvvchoice, aptitude-ability tests are administered: Verbal (28 items). 
Numerical (14 items), Mechanical-Electrical (14 items). Spatial (14 items). 
Abstractions (14 items) and Clerical (14 items). 

(4) Job Satisfaction Indicators - 180 items using a like-neutral-dislike scale 
measure temperaments in the following areas: Versatile, Repetitive, 
Performing Under Specific Instruction, Dominant, Gregarious, Isolative, 
Influencing, Self-Controlled, Valuative, Objective, Subjective and Rigor- 
ous. All of these are either directly or indirectly related to Tempeia- 
ments as defined in the Handbook for Analyzing Jobs. 

r 



Two other versions of WOWI exist - An English version with a fifth grade 
reading level and a Spanish language version. 
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b. Grouping of Work Samples - The tests are arranged in the order described 
immediately above. Each part of the system can be administered separately. 

c. Manual - The Interpretation Manual and Cn ide to Career Families contains 
general information on use and detailed information on the fairly complex- 
interpretation process needed to convert test scores to the 17 Career Families 
listed above, and the relationship of the WOWI to other tests. Other ad- 
ministration information is located on the answer sheet. Although tnc manual 
contains no information on the administration of tests and scoring of the 
tests, such information is available from the publisher. 

For computer administc- 1 tests, a separate introduction is given. It explains 
how to boot the disk and then refers the user to the respective computer 
manual for complete instructions. It also mentions how the client should 
return to an answer he/she would like to change. 

A step-by-step interpretation cassette tape is available for test administrators. 
Physical Aspects 

a. Packaging of the Work Samples - This section is not relevant for the WOWI. 

b. Durability - The test booklets arc rcuscable. 

c. Expendable Supplies - If computer administration is used, printer paper is the 
only expendable supply. The test booklets require a separate answer shcot 
for each client. Work sheets for each of the 17 Career Families are also 
expendable. 

d. Replacement - When administering WOWI by computer, the disk is preset by 
the developer for a certain number of clients. After the number of clients 
are run, the disk can be returned to be "reset," or a new disk can be or- 
dered. 

c. Computer Requirements - The software is contained on a single floppy disk. 
It requires an IBM-PC or IBM compatible computer with one disk driv and at 
least 128K RAM. The user must supply a DOS 2.0 or 2.1 disk to initialize the 
program. An 80 column serial or parallel printer is required. 

Vocational Evaluation Process 

a. Preliminary Screening - Although not str^ed in the manual, the evaluator will 
have to make certain that the client can read at about the eighth grade level 
prior to administration. 

b. Sequence of Work Sample Administration - WOWI is administered in the 
sequence described in 2.b. above. While it is possible lo vary administration 
when using the paper-and- -jncil version, the computer version permits no 
variation. 

c. Client Involvement - The final results, called the Inventory Profile, is de- 
signed for both client and evaluator use. Following test administration, the 
client can be actively involveu in translating the results into specific Career 
Families and DOT job titles. Although not stated in the manual, the Invcn- 
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tory Profile and the sequent process could provide for close interaction 
between client and ^valuator. 

d. Evaluation Setting - This is not specified in the manual; the instrument 
implies a formal testing situation. The test book and answer sheet may be 
sent home with the test taker if the administrator thinks it is appropriate. 

e. Time to Complete the Entire System - The WOWI tests and inventories are 
pure power tests, i.e., they are not timed. It is estimated by the reviewer 
that the WOWI could be given in between 45 minutes and an hour and a half. 
Interpretation would take longer. 

Administration 

a. Procedures - No administration procedures are given in the manual. 

b. Method of Instruction Giving • Two separate metho'^*' are implied: (1) In 
computer administration, all instructions are given c he screen, (2) the test 
booklet can be self-administered or f^upervised by an evaluator. It must be 
noted that even when the computer version is used^ many of the test items 
arc still prese-^ted in the test booklet and NOT on the computer screen. This 
requires the client to be constantly moving his/her eyes between the booklet 
and the computer screen. The test booklet is used only on those sections 
which include diagrams or illustrations (i.e., 56 items). 

c. Separation of Learning/Performance - This is minimal; there are few or no 
practice exercises for the tests. Because the individual tests are very short, 
*his is a very serious problem. 

d. Providing Assistance to the Client - No procedures are given in the manual. 

e. Repeating Work Samples - This is not relevant for WOWI. 
Scoring and Norms 

Because the WOWI is jot a work sample system, the outline for this section does 
not apply. The WOWI is a computer scored test and the results are printed in a 
profile. If the test is computer administered, the computer immediately scores the 
test and prints the results. If the paper-and-pencil version of the test is ad- 
ministered, it is mailed to the developer for scoring and profiling. 

Oby er'^ ation of Clients 

There no information in the manual about the observation of clients. Because 
administration is tormal and the WOWI is completed in a relatively short time, 
opportunity for client observation is limited. 

Re porting 

a. Forms - A separate answer sheet is used for paper-and-pencil administration. 

b. Final Report Format - There are two report formats. The locally scored 
computerized version results in an 80 column format report. The report 
format is well-designed and easy for both client and evaluator to read. The 




contents are as follows: (1) demographic information, (2) A graph containing 
the score for each of the 17 Career Families is presented with their DOT 
occupational division. Each interest is presented on a scale from -60 to +60, 
with 0 as E neutral point. The percentage of dislike, neutral and like respon- 
ses are given. A short verbal explanation is also printed. (3) The Vocational 
Training Potentials presents the scale scores in percentiles and T scores for 
each of the six ability tests. The scales are adjusted for the appropriate 
norm group. Verbal definitions of each aptitude are given. (4) The Job 
Satisfaction Indications are presented in graphic format, ranging from -60 to 
+60, with 0 as a neutral point. The percentage of dislike, neutral and like 
responses are given. Each temperament is defined. (5) The final page is a 
summary, containing the two most liked GOE interest areas, two best liked 
school subjects, the two highest Career Families, the two highest aptitudes, 
and the two highest temperaments. 

For users sending in the test results for scoring, a single page format con- 
taining the following information is used: (1) Career Interest Activities, (2) 
Demographic Data, (3) Job Satisfaction Indicators, (4) Vocational Training 
Potentials, and (5) Summary. 

9. Utility 

a. Vocational Exploration - Although there i? little direct experience to be 
gained by the client, the WOWI could serve as a first step of a vocational 
^..ploration program. The client uses the summary information in the sum- 
mary report to locate specific DOT titles. However, no vocational exploration 
inTormavion is contained in the system per se. 

b. Vocational Recommendations - Recommendations are presented in the form of 
test results. Specific recommendations are not contained on the printout. 

c. Counselor Utilization - The system offers a well-balanced combination of 
interests, aptitudes and temperaments. In vocational evaluation, V/OWI ap- 
pears to have its p mary utility as an initial screening device, to be followed 
by detailed occupational exploration, physical capacities assessment and a 
detailed measurement of other aptitudes and abilities. 

10. Training in the System 

a. Training Required - No information available; assume none. 

b. Training Available - A cassette training tape is available. Occasional scoring 
and interpretive information is sent to users in mailingo. On-site training is 
also available for a fee. 

c. Duration - No information available. 

d. Follow-up - No information available. 

1 1. Technical Considerations 

a. Norm Base - Due to central scoring procedures, normative data are collected 
on an ongoing basis. This results in a constant updating of norms. Printed 
normative data Tor the aptitude tests are available on the following popula- 
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tions: business, junior college, public high school, private high school, male 
prisoners, technical school, vocational counseling and vocational rehabilitation 
clients. Within each group, norms are presented for several levels of educa- 
tion and age groupings. Sample sizes within each of these groups range from 
five to several hundred. Un'^ortunately, there is no available information on 
the selection of the normative groups. 

b. Reliability - Reliability is presented in terms of alpha coefficients, Spearman- 
Brown split-half and •'tandard errors of measurement. Individual scale alphas 
range from .24 to .94; most of these are in the .80*s. Over 90% of these are 
significant at the .001 level. 

c. Validity - The only validity information presented in the manual listed cor- 
relations between Career Family and the Job Satisfactory scores and the 
following Strong-Campbell Interest Inventory scales: General Occupational 
Themes, Basic Interest Scales, and Occupational Scales. Correlations between 
the Kuder DD Occupational Interest Occupal'onal Scales and College Scales, 
Edwards Personal Preference Schedule scoi 3, 16 Personal Factors Test, 
Taylor-Johnson Temperament Analysis Profile Scores, and Study of Values and 
the WOWI were also presented. Because no information is provided on samples 
and testing conditions, these data are hard to interpret. The manual contains 
no information on predictive and concurrent validity. 

12. Reviewer*s Summary and Comments 

The WOWI is designed to provide an assessment of three major considerations in 
vocational planning and hiring: interests, aptitudes and temperaments. The WOWI 
combines these three variables types into one system; it then uses the results as 
the beginning of a vocational planning process. Perhaps the most unique feature 
of the WOWI is the specific measurement of the temperaments as defined in the 
^landbook for Analyzing Jobs . Although temperaments are widely used in voca- 
tional planning and in computerized job matching systems (Botterbusch, 1986), the 
reviewer knows of no specific tests or instruments that have been developed other 
than this section of the WOWI. 

The WOWI can be used as a screening device in the very beginning of the voca- 
tional evaluation process. The WOWI would provide a quick first measure of basic 
aptitudes, interest?? ad temperaments. 

The WOWI does have several problems: First, there is a lack of administration 
and scoring instructions in the manual. Second, even with the computer ad- 
ministration the tesc items for some sections are taken from the printed test 
booklet. It should be pointed out, however, that the items not presented on the 
screen are only those containing diagrams, charts and illustrations. Third, the 
aptitude tests are very short, many containing only 14 iiems. Fourth, because 
these tests are pure power tests, they increase in difficulty very rapidly. It 
would be interesting to have the item analy.sis data for thei:e scales. Fifth, the 
reported validity is limited to correlations with other interest and personality 
tests. 

In summary, in vocational evaluation the WOWI has use as a screening measure 
early in the evaluation process. The most unique feature is the direct measure- 
ment of DOT related temperaments. 
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13. Addre ss 



World of Work, Inc 
2923 N. 67th Place 
Scottsdale, AZ 85251 

14. Cost 

World of Work Inventory Test Booklet (reusable): 



1 to 5 $2.00 each 

6 to 24 $1.50 ^ach 

25 to 49 $1.25 each 

over 30 $1.00 each 

WOWI Answer Sheets (includes scoring): 

1 to 4 $4.25 each 

5 to 24 $3.50 each 

25 to 99 S3.00 each 

100 to 400 $2.50 each 

WOWI Floppy Disk 

10 test administrations (plus 1 test booklet) $34.00 

25 test administrations (plus 1 test booklet) $73.50 

50 test administrations (plus 1 test booklet) $146.00 

Manuals and Related: 

Interpretation Manual and Guide to Career Families $ 1 4.95 

Mini Manual $2.75 

Using the DOT for Job Placement $22.50 

Occupations in Demand 1935 - 20 0Q $25.00 

Other Materials 

Occupational Exploration Worksheets set of ! 7 $1.50 

Packages of 50 V/orksheets $3.50 

Specimen Set: Inventory Booklet, 1 answer sheet, 

Interpretation manual ?.nd Guide to Career Famines $19.95 

Instructional Cassette Tape $10.00 
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Glossary of Vocational Evaluation Terms 



Aptitudes - The specific capacity or ability required of a person in order to facilitate 
the learning of a job or task. Job analysis us'ng the U.S. Department of Labor 
method collects data on 11 aptitudes: G - General Learning Ability; V - Verbal; N- 
Numerical; S - Spatial Perception; P - Form Perception; Q - Clerical Perception; K- 
Motor Coordination; F - Finger Dexterity: M - Manual Dexterity: E - Eyc-Hand-Foot 
Coordination; and C - Color Discrimination. The first nine of these are commonly 
measured by the General Aptitude Test Battery. 

Assessment - The " 'process of finding out what the strengths and limitations of an 
individual are in terms of optional functional outcome and developing proposals for 
alternate service plans.' Assessment is to rehabilitation services what diagnosis is to 
medicine. While vocational assessment uses many of the same techniques and methods 
as vocational evaluation, assessment is a more one-way process where clients' voca- 
tional strengths and weaknesses are determined by an evaluator or teacher. Because 
it deals with the basics of client capabilities, physical limitations, vocational relevant 
aptitudes and functional literacy, assessment is more limited in scope than evaluation. 
Assessment usually has little concern foi vocational exploration, interests and at- 
titudes and tends to confme itself of the immediate skills available" (Botterbusch, 
1983, p. 8). 

Career - "The sequence of occupations, jobs, [and] positions throughout a person's 
working life. The sequence of events in the life of a person as he [sic] in a job or 
as he [sic] changes from one job to another in the occupational structure" (VEWAA, 
1983, p. 2). 

Cluster Trait Sample - "A work sample that assesses a number of traits inherent in a 
group of related jobs. Based upon an analyses of occupational grouping and the traits 
necessary for successful performance, it is intended to assess the client's potential to 
perform jobs that have a common set of performance requirements" (VEWAA, 1983, p. 
2). 

Bi-Polar laterests - See Interests. 

Data-People-Things (DPT) - The middle three digits of the DOT Code, indicates the 
complexity of a job in relation to three separate hierarchies of Data, People and 
Things. 

Disadvantaged - A person constrained by reason of youth, advanced age, low educa- 
tional attainment, ethnic or racial factors, prison or delinquency records, or any other 
condition, especially in association with poverty. 

Environmental Conditions - The physical surroundings of the job that make specific 
demands on the worker's physical capacity. The Handbook for Analyzing Jobs includes 
the following factors: work location (i.e., inside, outside or both); extreme cold; 
extreme heat; wet and/or humid; noise and/or vibration; hazards and atmospheric 
conditions. 

Functional Limitations - Restrictions of physical or mental functions that hinder a 
person's ability to take care of his/her own affairs. 
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Genera! Educational Development (GED) • Those aspects of both formal and informal 
education contributing to the worker's Reasoning, Mathematical, and Language skills. 
These are considered the basic skills needed to some degree in all jobs. Reasoning 
and Mathematical are divided in six areas; language into five. 

Guide to Occupational Exploration Codes (GOE) - See Interests. 

Industrial Standards - "Actual worker requirements from industry based on the expec- 
tations of the employer in terms of quality, quantity, and work behaviors" (VEWAA, 
1983, p. 6). 

Interests - Interests are a liking or preference for an activity. There are two methods 
for classifying interests in the DOL job analysis methodology. The first is using the 
codes given in th'^ Guide to Occupational Exploration : the 12 general classifications 
are: Artistic, Scientific, Plants and Animals, Protective, Mechanical, Industrial, Busi- 
ness Detail, Selling, Accommodating, Humanitarian, Leading-Influencing, and Physical 
Performing. The second method is to use the bi-polar interest factors; if one of the 
pair is selected the other, by definition, cannot be chosen. There are five pairs of 
factors: things and objects vs. communication of data; business contact with people \s. 
scientific and technical activities; routine, concrete, organized activities vs. abstract 
and creative activities; presumed good of people vs. tangible production. 

Job Analysis - The systematic study of a position in term of what the worker does, 
how he/she does it, and what the outcomes are. Although there are numerous meth- 
ods of job analvsis, the most commonly used in vocational evaluation are the proce- 
dures developed by the U.S. Department of Labor. (DOT, 1972; DOL, 1983) 

Job Clusters - **Related occupations grouped on the basis of similar job requirements, 
such as specific duties of the job, materials and equipment used, skill and knowledge, 
and worker characteristics required" (VEWAA, 1983, p. 6). 

Job Exploration - "A process whereby an individual is exposed to work experience and 
occupational information inter ded to increase his [sic] knowledge of the world of 
work" (VEWAA, 1983, p. 7). 

Job Site Evaluation - *'An evaluation technique where the client performs the actual 
job duties in a real work situation. Performance is supervised and evaluated by the 
employer in coordination v/ith evaluation staff (VEWAA, 1983, p. 8). 

Job Task - See Task. 

Occupational Groups Arrangement (OGA) - The first three numbers of the DOT code. 
These are three levels of grouping of jobs from general to specific. All jobs in the 
DOT are grouped into nine major Categories, 82 two digit Divisions, and 559 three 
digit Groups. 

Physical Demands - These are the physical requirements made of the worker by the 
specific job-worker situation. There are two separate classification systems for 
physical demands (i.e., A Handbook for Analyzing Jobs and A Guide to Job Analvsis) 
The Handbook includes six separate factors: Strength (sedentary, light, medium, heavy 
and ver> heavy); climbing and/or balancing; stooping, kneeling, crouching, and/or 
crawling; reaching, handling, fingering, and/or feeling, talking and/or hearing; and 
seeing. In some job matching and work sample systems, strength is sometimes treated 
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as a separate variable. Physical demands are also called "Physical Limitations." The 
Quide contains 285 separate physical demands. 

Prevocational Evaluation - "An assessment process conducted prior to work or training 
to determine if an individual has the ability to develop and to maintain work skills 
and related work behaviors. The term, which originated in education and occupational 
theraf >, is used primarily with individuals having little or no work history (i.e., stu- 
dents)" (VEWAA, 1983, p. 8). 

Single Trait Work Samples - "Assesses a single trait or characteristic. It may have 
relevance to a specific job or many jobs, but it is intended to assess a single, isolated 
factor" (VEWAA, 1983, p. lO). In form and structure a single trait work sample is 
very similar to a standardized performance test. 

Situational Assessment - "A systematic observation process for evaluating work-related 
behaviors in a controlled work environment..." (VEWAA, 1983, p. 10). Most situational 
assessment is conducted in sheltered employment settings. 

Specific Vocational Preparation (SVP) - The amount of time required to learn the 
techniques, acquire information and develop the facility needed for average perfor- 
mance in a specific job-worker situation. This training may be acquired in a school, 
work, military, institutional, or a vocational environment. There are nine levels of 
SVP, ranging from a "short demonstration only" to "over 10 years." 

Strength - See Physical Demands. 

Task - "A grouping of the elements and work activities of a job that have a common 
purpose, and are closely related in terms of methodologies, materials, products, ser- 
vices and types and sequences of worker actions. (DOL, 1982, p. 6). 

Temperaments - The adaptability requirements made on the worker by specific types 
of jobs. Like physical demands and environmental conditions, there are two slightly 
different lists of temperaments. The following temperaments are from the Handbook 
for Analyzing Jobs: ' erpretation of feelings; influencing people; making evaluations 
on sensory or judgmental criteria; responsibility for direction; making decision on 
measurable criteria; dealing with oeople; repetitive; performing under stress; precise 
attainment of set limits; and variety of duties. 

Vocational Counseling - "Process of assisting a person to understand vocational liabil- 
ities and assets, and providing occupational info* .nation to assist them in choosing an 
occupation suitable to their interests and liabilities" (VEWAA, 1983, p. 12). 

Vocational Evaluation - "A comprehensive process that systematically utilizes work, 
either real or simulated, as the focal -^oint for assessment and vocational exploration, 
the purpose of which is to assist inowiduals in vocational development Vocational 
evaluation incorporates medical, psychological, social, vocat^nal educational, cultural, 
and economic data into the process to attain the goals of evaluation" (VEWAA, 1983, 
p. 12). 

Vocational Exploration - See Job Exploration. 

♦Vork Behaviors - "Those aspects of behavior in a work setting that enable a person 
to meet the demands of his [sic] job in accordance with employment standards. This 
includes such areas as: attendance, punctuality, hygiene, social behavior, team work, 
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cooperation, ability to accept constructive criticism, ability to accept supervision, 
effort..." (VEWAA, 1983, p. 13). 



Work Sample - "A well defined work activity involving tasks, materials and tools 
which are identical or similar to those in an actual job or cluster of jobs. It is used 
to assess an individual's vocational aptitude, worker characteristics, and vocational 
interests" (VEWAA, 1983, p. 14). 

Worker Trait Profile - The combination of the values assigned to all or some of the 
following variables: GED, SVP, aptitudes, physical demands, bi-polar interests, GOE 
interests, environmental conditions or temperaments. 
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Philadelphia Jewish Employment and Vocational Service 

Work Sample System 



(JEVS) 



1. Development 

a. Sponsor - Originally developed for the Manpower Administration of the U.S. 
Department of Labor for use in WIN and CEP programs, the JEVS has been 
refined by the Philadelphia Jewish Employment and Vocational Service. 



b. Target Group - Initi?ny designed for the disadvantaged, the system has been 
used in the last several years as an assessment device for special needs 
populations. 

c. Basis of the System - The present basis is the Work Group system of the 
fourth edition of the Dictionary of Occupational Titles and the 1979 Guide for 
Occupational Exploration (GOE). The philosophical basis is a trait-factor 
approach between common aptitudes and behavioral demands of the Work 
Groups and work samples. 



2. Organization 



a. Name and Number of Work Samples - The JEVS contain 28 different work 
samples. (The purchaser receives a total of 48 separate work samples, 20 of 
which are duplicates of the most used work samples.) The 28 work samples 
are referenced to 12 Work Groups. Most of the 28 work samples are used in 
more than one Group: 



05.03 - Engineering Technology - Condensing Principle 

05.05 - Craft Technology - Blouse/Vest Making, Pipe Assembly, Resistor 
Reading, Nail and Screw Sort, Lock Assembly, Telephone Assembly 

05.09 - Material Control - Computing Postage, Nail and Screw Sort, Filing by 

Numbers 

05.10 - Crafts - Resistor Reading, Telephone Assembly, Metal Square Fabrica- 

tion, Ladder Assembly, Union Assembly 
05.12 - Elemental Work Mechanical - Hardware Assembly, Grommet Assembly 

06.02 - Production Work - Telephone Assembly, Hardware Assembly, Metal 

Square Fabrication, Grommet Assembly 

06.03 - Quality Control - Nail and Screw Sort, Collating Leather Samples, Nut 

Packing, Tile Sorting 
06.041 - Elemental Work: Industrial - Belt Assembly, Grommet Assembly, 

SJgii Making, Budgette Assembly 
06.0411 - Elemental Work: Industrial - Collating Leather Samples, Nut Pack- 
ing, Washer Threading, Nut, Bolt and Washer Assembly 

07.02 - Mathematical Detail - Computing Postage, Payroll Computation, Ad- 

ding Machine 

07.03 - Financial Detail - Computing Postage, Payroll Computation, Adding 

Machine 

07.05 - Records Processing - Filing by Letters, Proofreading, Filing by Num- 
bers 

07.07 - Clerical Handling - Filing by Numbers, Rubber Stamping 
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b. Grouping of Work Samples - The work samples are organized into 12 Work 
Groups for reporting and interpretation purposes. 

c. Manual - The Work Sample Evaluator's Handbook contains detailed administra- 
tion and scoring instructions as well as numerous photographs to illustrate 
proper setup and common errors. The manual is well written and easy to 
follow. 

Physical Aspects 

a. Packaging of the Work Samples - Each work sample is packaged individually; 
no tools or parts are shared with other samples. 

b. Durability • The system uses common tools and materials that should be very 
durable. The one exception is the telephone. 

c. Expendable Supplies - In addition to referral, report, and other forms, the 
major expendable supplies are: fabric, paper pads, sheet metal, and string. 
While these supplies should be available locally, they can also be purchased 
from the developer. 

d. Replacement - Most tools and equipment can be locally purchased; other items 
(e.g., colored chips) are available from the developer. 

e. Computer Requirements - Not applicable. 
Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The work samples are administered 
in order of complexity beginning with Nuts, Bolts, and Washer Assembly and 
ending with Condensing Principle Drawing. If a client is obviously not able 
to complete the work samples at any one level, more complex work samples 
are usually not administered. 

c. Client Involvement - A client orientation is given at the beginning of work 
sampling, a motivational group interview at the end of the first day and a 
structured Feedback interview at the completion. Since work sample ad- 
ministration resembles realistic work setting, interaction between client and 
evaluator occurs between work sample administration and during the above 
sessions. 

d. Evaluation Setting - A realistic work atmosphere and setting are stressed in 
the manual. 

e. Time to Complete the Entire System - The average client takes six or seven 
days to complete the 28 work samples. 

Administration 

a. Procedures - The layout is clearly described and photographs are used to 
insure proper setup. The materials listed for each work sample are not iMcd 
at the beginning of the instructions for that work sample. The evaluator is 
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provided with a list of materials for each work sample as detailed in the set- 
up instructions. 

b. Method of Instruction Giving - All instructions are oral and some demonstra- 
tion. Reading is required of the client only when it is a requirement in the 
job area being sampled. 

c. Separation of Learning/Performance - Most of the work samples do not have 
a separate practice period. Typically, ihe evaluator gives the instructions 
while providing a demonstration. The client attempts the task without a prior 
period of practice. There are no set criteria to be met prior to timing. 
Thus, there is minimal separation of learning from performance. 

d. Providing Assistance to Clients - Assistance can be civcn after the initial 
instruction period; but this results in lowering the client's final score. The 
manual contains detailed procedures for providing assistance and describes 
three levels of helping. Each level and each type are clearly defined. This 
emphasis of the analysis of the type of assistance is unique to the JEVS 
system. 

e. Repeating Work Samples - Readministration is not recommended because it 
invalidates results. 

6. Scoring and Norms 

a. Timing - A time clock is used to stamp the starting and stopping time for 
each wcrk sample. A separate time stamp slip is used for each work sample. 

b. Timing Interval - The evaluator punches the time clock after instructions are 
given and the client punches the clock when the work sample is completed. 

c. Time Norms - Time results are rated on a three-point scale based on the 
number of minutes to completion. The scale is taken from percentile scores. 

d. Error Scoring - Most work samples use a random check of items that are 
compared to carefully defined scoring criteria; many use photographs to 
illustrate quality standards. Assistance points are also incorporated into the 
error scoring procedures. 

e. Scoring Aids - Minimal use is made of scoring aids. 

f. Quality Norms - Quality is rated using a three ; . *nt scale based on the 
number of counted errors. 

g. Emphasis in Scoring - Time and quality are given equal weight. 

7. Observation of Clients 

a. Work Performance - Sixteen specific factors (e.g., size discrimination, forn. 
perception) and four more general factors (e.g., accuracy, neatness) are 
specified for the system; each work sample has certain factors listed that are 
to be observed. The system stresses the recording of accurate behavioral 
observations. 




b. Work Behaviors - The system carefully lists and defines many work related 
behaviors that arc to be carefully observed. For example, in writing observa- 
tions about communication, articulation, tone of voice and grammatical usage 
are to be noted. Some other behaviors are cooperativ^^ness with co-workers 
and supervisors, r^^action to criticism, and frustration tolerance 

c. Recording Systems - Many of the work performance factor*: are rated on a 
three-point scale, with all ratings clearly defined and illustrated. 

d. Frequency of Observation - The system uses extensive obsei nations. Observa- 
tion of defined work factors is required for each work sample; these are 
summarized daily. 

8. R eporting 

a. Forms - Standardized forms are included for: reporting the results of each 
work sampler, daily observation summary, feedback interview and a final 
report. 

b. ^'inal Report Format - The well organized standardized format includes some 
ranking of work sample performance, recommended Work Groups and rationale, 
and extensive written comments on performance and beha* ior. 

9. Utility 

a. Vocational Exploration - Client vocational exploration is seriously limited by 
two factors: (1) many of the work samples t^nd to be abstract, and (2) there 
is no orientation relating the work samples to jobs. 

b. Vocational Recommendations - The final report has a space fo. two Work 
Groups that are suggested for additional planning. The recommendations arc 
related to the fourth edition of the DOT and the GOE and are geared for 
both training and job placement. 

c. Counselor Utilization - The system and the final report are oriented toward 
the counsels*; however, counselor familiarity with the DOT and GOE is neces- 
sary for optimal counselor use. 

10. Training Required 

a. Training Required - Yes 

b. Training Available - Yes 

c. Duration - One week; usually held in Philadelphia. Regional training is 
available under certain conditions. 

d. Follow-up - Not available. 

11. Technical Consideration 

a. Norm Base - The system was renormed in 1975 on a total population of over 
1,100 clients in 32 facilities throughout the U.S. T^ne and quality norms are 
reported for the total sample as well as separ'^ce norms by sex, and for 
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